Java | Techlist.io

lineNov 7, 2025Original article

Code Quality Improvement Techniques Part 22: To equal, or not to equal (opens in new tab)

The post argues that developers should avoid overriding the `equals` method to compare only a subset of an object’s properties, as this violates the fundamental principles of identity and structural equivalence. Implementing "partial equality" often leads to subtle, hard-to-trace bugs in reactive programming environments where UI updates depend on detecting changes through equality checks. To ensure system reliability, `equals` must strictly represent either referential identity or total structural equivalence. ### Risks of Partial Equality in Reactive UI * Reactive frameworks such as Kotlin’s `StateFlow`, `Flow`, and Android’s `LiveData` utilize `distinctUntilChanged` logic to optimize performance. * These "observable" patterns compare the new object instance with the previous one using `equals`; if the result is `true`, the update is ignored to prevent unnecessary re-rendering. * If a `UserProfileViewData` object only compares a `userId` field, the UI will fail to reflect changes to a user's nickname or profile image because the framework incorrectly assumes the data has not changed. * To avoid this, any comparison logic that only checks specific fields should be moved to a uniquely named function, such as `hasSameIdWith()`, instead of hijacking the standard `equals` method. ### Defining Identity vs. Equivalence * **Identity (Referential Equality):** This indicates that two references point to the exact same object instance, which is the default behavior of `Object.equals()` in Java or `Any.equals()` in Kotlin. * **Equivalence (Structural Equality):** This indicates that two objects are logically the same because all their properties match. In Kotlin, `data class` implementations provide this by default for all parameters defined in the primary constructor. * Proper implementation of equivalence requires that all fields within the object also have clearly defined equality logic. ### Nuances and Implementation Exceptions * **Kotlin Data Class Limitations:** Only properties declared in the primary constructor are included in the compiler-generated `equals` and `hashCode` methods; properties declared in the class body are ignored by default. * **Calculated Caches:** It is acceptable to exclude certain fields from an equality check if they do not change the logical state of the object, such as a `cachedValue` used to store the results of a heavy mathematical operation. * **Context-Dependent Equality:** The definition of equality can change based on the model's purpose. For example, a mathematical model might treat 1/2 and 2/4 as equal, whereas a UI display model might treat them as different because they represent different strings of text. When implementing `equals`, prioritize full structural equivalence to prevent data-stale bugs in reactive systems. If you only need to compare a unique identifier, create a dedicated method instead of repurposing the standard equality check.

Java Android Kotlin Code Quality+3

lineSep 12, 2025Original article

Code Quality Improvement Techniques Part (opens in new tab)

When implementing resource management patterns similar to Kotlin's `use` or Java's try-with-resources, developers often face the challenge of handling exceptions that occur during both primary execution and resource cleanup. Simply wrapping these multiple failures in a custom exception container can inadvertently break the calling code's error-handling logic by masking the original exception type. To maintain code quality, developers should prioritize the primary execution exception and utilize the `addSuppressed` mechanism to preserve secondary errors without disrupting the expected flow. ### The Risks of Custom Exception Wrapping Creating a new exception class to consolidate multiple errors during resource management can lead to significant issues for the caller. * Wrapping an expected exception, such as an `IOException`, inside a custom `DisposableException` prevents specific `catch` blocks from identifying and handling the original error. * This pattern often results in unhandled exceptions or the loss of specific error context, especially when the wrapper is hidden inside utility functions. * While this approach aims to be "neat" by capturing all possible failures, it forces the caller to understand the internal wrapping logic of the utility rather than the business logic errors. ### Prioritizing Primary Logic over Cleanup When errors occur in both the main execution block and the cleanup (e.g., `dispose()` or `close()`), it is critical to determine which exception takes precedence. * The exception from the main execution block is typically the "primary" failure that reflects a business logic or IO error, whereas a cleanup failure is often secondary. * Throwing a cleanup exception while discarding the primary error makes debugging difficult, as the root cause of the initial failure is lost. * In a typical `try-finally` block, if the `finally` block throws an exception, it naturally suppresses any exception thrown in the `try` block unless handled manually. ### Implementing Better Suppression Logic A more robust implementation mimics the behavior of Kotlin’s `Closeable.use` by ensuring the most relevant error is thrown while keeping others accessible for debugging. * Instead of creating a wrapper class, use `Throwable.addSuppressed()` to attach the cleanup exception to the primary exception. * If only the primary block fails, throw that exception directly to satisfy the caller's `catch` requirements. * If both the primary block and the cleanup fail, throw the primary exception and add the cleanup exception as a suppressed error. * If only the cleanup fails, it is then appropriate to throw the cleanup exception as the standalone failure. ### Considerations for Checked and Unchecked Exceptions The impact of exception handling varies by language, particularly in Java where checked exceptions are enforced by the compiler. * Converting a checked exception into an unchecked `RuntimeException` inside a wrapper can cause the compiler to miss necessary error-handling requirements. * If exceptions have parent-child relationships, such as `IOException` and `Exception`, wrapping can cause a specific handler to be bypassed in favor of a more generic one. * It is generally recommended to only wrap checked exceptions in `RuntimeException` when the error is truly unrecoverable and the caller is not expected to handle it. When designing custom resource management utilities, always evaluate which exception is most critical for the caller to see. Prioritize the primary execution error and use suppression for auxiliary cleanup failures to ensure that your error-handling remains transparent and predictable for the rest of the application.

Java Kotlin Code Quality Resource Management+2

airbnbAug 13, 20254 min readCurated summary

Migrating Airbnb’s JVM Monorepo to Bazel

Airbnb migrated its tens-of-millions-of-lines JVM monorepo from Gradle to Bazel over 4.5 years, achieving faster builds, testing, IntelliJ syncs, and development deployments. The move was driven by Bazel’s scalable remote execution, hermetic builds, and ability to provide shared infrastructure across Airbnb’s language-specific repositories. A gradual rollout, extensive automation, and close collaboration with service teams were central to making the migration successful. ## Results of the Migration - Build CSAT increased from 38% to 68%. - Local build and test times became 3–5 times faster. - IntelliJ syncs became 2–3 times faster. - Development-environment deployments became 2–3 times faster. ## Why Airbnb Chose Bazel ### Faster Builds Through Remote Execution - Large Gradle builds frequently took more than 20 minutes locally, while pre-merge CI builds had a p90 of 35 minutes. - Gradle had already been optimized with powerful machines and build sharding, but sharding caused underutilization and duplicated shared work. - Bazel’s cacheable actions and remote build execution enabled thousands of actions to run in parallel on short-lived workers. - “Build without the Bytes” reduced the amount of build output developers needed to download. - Bazel analysis runs in parallel, unlike the often single-threaded configuration phase of large Gradle projects. - Remote execution also improved local build performance, not just CI performance. ### More Reliable and Reproducible Builds - Gradle tasks could access the entire filesystem, creating accidental dependencies and race conditions. - Bazel sandboxes expose only declared inputs to each action, preventing undeclared files from affecting builds. - Bazel’s remote execution runs actions in identical containers with strict resource limits. - Using remote execution for both local and CI builds reduced differences between developer and CI environments. ### A Shared Build Infrastructure Layer Because Airbnb’s web, iOS, Python, Go, and JVM repositories all use Bazel, the company could standardize infrastructure for: - Remote caching - Remote build execution - Affected-target calculation - Build Event Protocol instrumentation and logging ## Starting with a Proof of Concept - Airbnb first migrated Viaduct, a large GraphQL monolith platform. - Viaduct was selected because it was complex, had slow builds, affected roughly 300 product engineers monthly, and had an infrastructure team willing to collaborate. - Bazel and Gradle initially coexisted, allowing developers to choose either system. - The team ported Viaduct’s build logic and created an automated Bazel build-file generator because the Gradle dependency graph continued to change. - Although Bazel was initially 2–4 times faster locally, developers did not adopt it immediately. - The team spent several additional months fixing missing integrations and bugs before Viaduct engineers voluntarily switched. ## Scaling Across the JVM Monorepo - Airbnb expanded breadth-first, aiming to make the entire repository compile and test under Bazel. - Gradle and Bazel continued to coexist during the migration. - This allowed developers to use Bazel locally while deployments still relied on Gradle. - Gradle provided a fallback when Bazel infrastructure, such as remote caching or execution, experienced incidents. - Maintaining two build graphs was costly, so Airbnb invested heavily in automation rather than requiring developers to maintain Bazel files manually. ## Automated Build-File Generation - The generator was inspired by Gazelle but was built internally to meet stricter performance requirements and handle dependency cycles. - It parses Java, Kotlin, and Scala source files to identify packages, imports, and symbol declarations. - These relationships are used to construct a file-level dependency graph. - Since generation ran on every commit before merging, Airbnb added external caching to keep it fast. - CI publishes a cached repository index for each mainline commit, allowing the generator to rescan only directories changed since that commit. Airbnb’s experience suggests that a large build-system migration is most effective when introduced incrementally: prove the benefits on a representative service, automate maintenance, preserve a fallback during rollout, and address developer workflow issues before expanding across the organization.

Read original

Java Kotlin Monorepo Jvm+4

datadogMay 23, 20243 min readCurated summary

How we migrated our static analyzer from Java to Rust

Datadog migrated its static analyzer from Java to Rust after finding that ANTLR-based parsing was too slow and language support was incomplete. Rust’s strong integration with Tree-sitter enabled broader language coverage, faster scans, and lower memory usage. The migration preserved behavioral parity while tripling performance and reducing memory consumption tenfold. ## Why Performance Became a Priority - Datadog runs analysis directly in customers’ CI environments, often on resource-constrained runners. - On a two-core, 7 GB GitHub Actions runner, medium repositories took about five minutes to scan instead of the target of under three minutes. - Codiga’s previous hosted environment used large, tuned servers, which masked some performance problems. - Java also required customers to use JVM 17+, potentially conflicting with JVM versions already installed in their CI environments. - Improving Java offered limited upside, so the team considered a rewrite despite its cost and risk. ## Static Analyzer Architecture - The analyzer consists primarily of: - A parsing layer that builds an abstract syntax tree (AST). - An execution layer that analyzes the AST, reports violations, and offers fixes. - Tree-sitter generates the AST. - The existing Java binding lacked important functionality, including Tree-sitter pattern matching. - Tree-sitter’s core libraries are implemented in Rust, where support was more complete. - Analysis rules are written in JavaScript and were originally executed through GraalVM’s polyglot capabilities. - Fast parsing, pattern matching, and rule execution were central to meeting the desired CI performance. ## Migrating from Java to Rust - Rust was selected because it is a first-class part of the Tree-sitter ecosystem and provided better access to its features. - The migration required: - Feature parity with the Java implementation. - Identical analysis results and reported violations. - No execution-time regressions. - Migrating the parser was relatively straightforward because Rust support came directly from Tree-sitter. - The Rust implementation: - Tripled analyzer performance. - Reduced memory usage by a factor of ten. - JavaScript execution moved from GraalVM to `deno-core`, a Rust-based V8 integration. - Only the core JavaScript functionality was included. - Disk and network capabilities were excluded because analysis rules do not need them, improving security. ## Migration Strategy and Rust Adoption - The team treated automated equivalence and performance tests as requirements for a successful rewrite. - Rust allowed the analyzer to integrate more directly with its key dependencies rather than maintaining a separate Java binding. - The broader migration also required replacing supporting Java components with corresponding Rust libraries; the article indicates that these mappings were documented as part of the transition. Overall, the move to Rust was justified by the analyzer’s deployment model: faster execution and lower resource consumption directly improved the experience of customers running scans in constrained CI environments.

Read original

Java Database Design CI/CD JavaScript+4

datadogMay 21, 20243 min readCurated summary

Engineering VP spotlight: Ivo Dimitrov

Ivo Dimitrov’s career evolved from low-level systems programming into engineering leadership focused on large-scale distributed storage. His experience at Microsoft and LinkedIn shaped his approach to building scalable data platforms, while Datadog attracted him with its talented people, modern technology, and culture of experimentation. Today, he leads Datadog’s Distributed Data Systems organization, supporting the company’s metrics, events, query, alerting, and analytics infrastructure. ## From Electrical Engineering to Systems Programming - Dimitrov initially studied electrical engineering and became interested in software while working on digital control systems. - His early work included contributing to a real-time operating system kernel. - He spent roughly a decade as an individual contributor, specializing in: - High-performance systems - Low-level programming - C and C++ - System software ## Transition from Individual Contributor to Manager - At Microsoft, Dimitrov worked on an early version of Azure Blob Storage. - Following a reorganization, he accepted an opportunity to lead his team despite having no prior management experience. - Microsoft supported the transition through: - Leadership mentorship - Formal management training - Guidance on communication, conflict resolution, and interpersonal leadership - He discovered that management allowed him to expand his ownership beyond individual projects and influence broader organizational outcomes. - The role combined his technical background with responsibilities such as cross-functional coordination, team development, and engineering strategy. ## Building Internet-Scale Storage at Microsoft and LinkedIn - At Microsoft, Dimitrov worked on storage systems supporting Hotmail. - After joining LinkedIn in 2014, he adapted to a technology environment centered on open source tools such as MySQL and Java. - He led development of Espresso, LinkedIn’s proprietary key-value storage platform. - The platform matured into a core system supporting approximately 95 percent of LinkedIn’s data sets. - He also helped oversee several other large-scale storage projects: - **Venice**, an open source platform for serving derived data - **Ambry**, an open source blob storage system - **Helix**, an open source cluster manager - These systems supported critical parts of LinkedIn’s internet-scale infrastructure. ## Why Datadog Was Appealing - Dimitrov was drawn to Datadog by three main factors: - Highly capable engineers and leaders - Interesting, modern technology - The opportunity to contribute to a rapidly growing company - Compared with the legacy systems and processes that had accumulated at LinkedIn, Datadog offered less bureaucracy and more freedom to: - Take thoughtful risks - Experiment - Deliver quickly - Fail fast and learn - Iterate and innovate - He was particularly interested in Datadog’s Kubernetes-based Metrics and Events platforms and the challenge of building best-in-class infrastructure during the company’s growth. ## Distributed Data Systems at Datadog - Dimitrov leads the Distributed Data Systems organization, which owns a portfolio of storage and data technologies. - Its responsibilities include: - **Metrics**, supporting metrics and time-series data - **Events**, handling semi-structured data such as logs, profiles, and traces - **Driveline**, a main-memory database optimized for online analytics - The **Cross-Platform Queries** team provides a unified query interface across systems that historically exposed separate, domain-specific APIs. - This reduces the learning curve for engineers and customers. - It abstracts the underlying data stores behind a common API. - The organization also operates Datadog’s Alerts platform, which generates a large share of the queries sent to the Metrics and Events systems. Dimitrov’s experience demonstrates how deep systems expertise can translate into effective engineering leadership. His recommendation by example is to remain technically engaged while expanding one’s scope—from writing individual components to shaping teams, platforms, and long-term engineering direction.

Read original

Java Database Design Kubernetes PostgreSQL+4

datadogSep 30, 2021Original article

How we optimized our Akka application using Datadog’s Continuous Profiler | Datadog (opens in new tab)

Datadog engineers discovered a significant 20–30% CPU overhead in their Akka-based Java applications caused by inefficient thread management within the `ForkJoinPool`. Through continuous profiling, the team found that irregular task flows were forcing the runtime to waste cycles constantly parking and unparking threads. By migrating bursty actors to a dispatcher with a more stable workload, they achieved a major performance gain, illustrating how high-level framework abstractions can mask low-level resource bottlenecks. ### Identifying the Performance Bottleneck * While running A/B tests on a new log-parsing algorithm, the team noticed that expected CPU reductions did not materialize; in some cases, performance actually degraded. * Flame graphs revealed that the application was spending a disproportionate amount of CPU time inside the `ForkJoinPool.scan()` and `Unsafe.park()` methods. * A summary table of CPU usage by thread showed that the "work" pool was only using 1% of the CPU, while the default Akka dispatcher was the primary consumer of resources. * The investigation narrowed the cause down to the `LatencyReportActor`, which handled latency metrics for log events. ### Analyzing the Root Cause of Thread Fluctuations * The `ForkJoinPool` manages worker threads dynamically, calling `Unsafe.park()` to suspend idle threads and `Unsafe.unpark()` to resume them when tasks increase. * The `LatencyReportActor` exhibited an irregular task flow, processing several hundred events in milliseconds and then remaining idle until the next second. * Because the default dispatcher was configured to use a thread pool equal to the number of processor cores (32), the system was waking up 32 threads every second for a tiny burst of work. * This constant cycle of waking and suspending threads created massive CPU overhead through expensive native calls to the operating system's thread scheduler. ### Implementing a Configuration-Based Fix * The solution involved moving the `LatencyReportActor` from the default Akka dispatcher to the main "work" dispatcher. * Because the "work" dispatcher already maintained a consistent flow of log processing tasks, the threads remained active and did not trigger the frequent park/unpark logic. * A single-line configuration change was used to route the actor to the stable dispatcher. * Following the change, the default dispatcher’s thread pool shrank from 32 to 2 threads, and overall service CPU usage dropped by an average of 30%. To maintain optimal performance in applications using `ForkJoinPool` or Akka, developers should monitor the `ForkJoinPool.scan()` method; if it accounts for more than 10–15% of CPU usage, the thread pool is likely unstable. Recommendations for remediation include limiting the number of actor instances, capping the maximum threads in a pool, and utilizing task queues to buffer short spikes. The ultimate goal is to ensure a stable count of active threads and avoid the performance tax of frequent thread state transitions.

Java Datadog Scala Akka+4

datadogSep 30, 20212 min readCurated summary

How we optimized our Akka application using Datadog’s Continuous Profiler

Datadog discovered that an unexpected 20–30% CPU overhead came from Akka’s use of `ForkJoinPool`, not from the log-processing code they initially suspected. Profiling showed that an actor handling intermittent latency metrics repeatedly caused worker threads to park and unpark. Moving that actor to a busier, more stable dispatcher reduced CPU usage by about 30%. ## How Profiling Revealed the Problem - Datadog used Akka to parallelize log-event processing through actors and dispatchers. - An optimization to log parsing produced little improvement, despite reducing parsing CPU time. - Continuous Profiler flame graphs showed increased CPU time in: - `ForkJoinPool.scan()` - `Unsafe.park()` - Thread-level analysis revealed that the default Akka dispatcher—not the expected dedicated work pool—was responsible. - Many of the affected threads were executing a latency-reporting actor. ## Why `ForkJoinPool` Was Consuming CPU - `ForkJoinPool` dynamically manages worker threads: - It creates threads when work increases. - It suspends idle threads with `Unsafe.park()`. - It resumes them with `Unsafe.unpark()`. - It terminates idle workers after a default period. - The latency actor received a few hundred events per second, processed them within milliseconds, and then remained idle until the next batch. - Because the pool allowed up to 32 threads—matching the number of processor cores—it repeatedly activated and suspended many workers. - These frequent parking and unparking operations created short CPU spikes and excessive time in `ForkJoinPool.scan()`. ## The Dispatcher Change - The team moved the latency actor from Akka’s default dispatcher to the main `work-dispatcher`. - The work dispatcher already handled a steadier stream of log-processing tasks, keeping its worker threads active. - This required only a configuration change assigning the actor to `work-dispatcher`. - CPU usage fell by roughly 30% across services. - The default dispatcher also shrank from 32 threads to 2, confirming that unnecessary thread activation was the cause. ## Recommendations - Monitor CPU time spent in `ForkJoinPool.scan()`, especially when it exceeds roughly 10–15%. - Limit the number of Akka actor instances. - Set a suitable maximum thread count for each pool. - Reduce the number of separate thread pools where practical. - Use task queues to absorb frequent, short-lived workload spikes. - Aim to keep the number of active `ForkJoinPool` workers relatively stable and avoid repeated parking and unparking.

Read original

Java Database Design Profiling Scala+4

datadogOct 7, 20203 min readCurated summary

How we wrote a Python profiler

Datadog built a Python continuous profiler because Python lacked Java-style, always-on production profiling tools. The post argues that deterministic profilers such as `cProfile` impose too much overhead for continuous use, while statistical profiling can provide representative performance data with minimal disruption. Datadog’s profiler addresses this through modular collectors, recording, scheduling, and data export. ## Profiling Versus Tracing - **Profiling** measures resource consumption such as CPU time and memory allocation to reveal performance problems. - **Tracing** records individual operations—such as SQL queries or HTTP requests—within a request timeline. - Tracing explains request latency, but profiling provides deeper insight into code-level execution and operating-system resource usage. ## Limitations of Deterministic Python Profilers - Python’s `cProfile`, available since CPython 2.5, records every function call and the time spent in each call. - It can provide a complete execution flow, but its usefulness depends heavily on code structure: - A program built around a few large functions produces little actionable detail. - A program containing thousands of functions can incur two- or three-times runtime overhead. - This overhead makes deterministic profiling unsuitable for always-on production environments. ## Why Profile in Production? - Optimizing without profiling is essentially guessing; real workloads often differ from development environments. - Production systems vary from developer machines in hardware, concurrency, input data, and workload behavior. - Continuous profiling captures how an application actually consumes resources under authentic conditions. - These requirements lead to statistical rather than deterministic profiling. ## Statistical Profiling in Python - Statistical profilers sample program activity periodically instead of recording every function call. - Individual short-lived calls may be missed, but repeated sampling over hours produces a reliable picture of resource consumption. - Lower overhead allows the application to run closer to its normal, unprofiled behavior. - Datadog evaluated numerous open-source Python profilers but found limitations involving platform support, collected data, or presentation-focused designs. - The team therefore developed its own statistical profiler, incorporating ideas from the tools it studied. ## Datadog Python Profiler Design The profiler was designed around three constraints: - Keep runtime overhead as low as possible. - Make deployment simple. - Support common operating systems and environments. Its architecture, inspired by the JDK Flight Recorder, consists of: - **Collectors:** Gather data such as CPU usage and memory allocation. - **Recorder:** Stores events produced by collectors. - **Exporter:** Sends profiling data outside the application. - **Scheduler:** Invokes components at appropriate intervals, such as exporting data every 60 seconds. - **Profiler:** Provides the high-level interface used by applications. ## Stack Collection - The stack collector is the primary built-in collector. - It wakes 100 times per second and captures the execution stack of every Python thread. - For each thread, it gathers information including: - The currently executing function - CPU time consumed - Exceptions being handled - The collector monitors the time required to inspect the application so it can control and limit its own CPU overhead. A statistical profiler with low overhead is the appropriate foundation for continuous production profiling, giving teams evidence about real application behavior without substantially changing that behavior.

Read original

Java Datadog Python Profiling+4