Build Systems | Techlist.io

slackNov 6, 20253 min readCurated summary

Build better software to build software better

Slack’s backend build pipeline for Quip and Slack Canvas once took 60 minutes, delaying feedback and slowing delivery. The team improved build performance by applying familiar software-engineering techniques—caching, parallelization, precise interfaces, and careful decomposition—using Bazel. The central argument is that build systems should be designed like high-performance programs: do less work, distribute unavoidable work, and define work units rigorously. ## Modeling Builds as Dependency Graphs - Applications can be represented as directed acyclic graphs of source files, intermediate artifacts, and deployable outputs. - A backend artifact depends on Python files, while a frontend artifact depends on TypeScript files. - Changing a Python file should rebuild the backend but not unrelated frontend components. - Clearly defined graph nodes allow build systems to optimize work rather than rebuilding everything. ## Caching and Hermetic Work - Caching avoids repeating expensive operations by storing outputs for known inputs. - The article uses a cached recursive `factorial()` function as an analogy: - The input is the cache key. - The return value is the cached artifact. - Effective caching requires work to be: - **Hermetic:** dependent only on explicitly provided inputs. - **Idempotent:** producing the same output for the same inputs. - Cache hit rate matters: poorly defined work units produce more cache misses. ## Granular Cache Units - Caching an entire `process_images(images, transforms)` operation is inefficient because changing one image invalidates the result for every image. - A more granular design caches `process_image(image, transform)` independently. - The higher-level operation can then reuse cached results and process only new image-transform combinations. - Smaller, well-defined units generally improve cache reuse and reduce rebuild time. ## Parallelizing Independent Work - Image processing can also be distributed across CPU threads using `ThreadPoolExecutor`. - Parallel work requires: - Completely specified inputs and outputs. - The ability to transfer data across thread, process, or network boundaries. - Handling completion and failure in any order. - APIs must document ordering guarantees; the threaded example returns images in completion order rather than input order. - Work-unit granularity affects scalability: - Too few large tasks limit available parallelism. - Too many tiny tasks may introduce coordination overhead. - The appropriate balance depends on the workload. ## Applying These Principles to Bazel - Bazel represents builds as directed acyclic graphs made of targets. - Each target defines: - Its input or dependency files. - Its output files. - The commands that transform inputs into outputs. - This structure provides the foundation for caching and parallel execution, just as explicit function inputs and outputs enable those optimizations in application code. The practical recommendation is to design build steps as small, hermetic, idempotent, and independently executable units. Combined with Bazel’s dependency graph, this lets teams avoid unnecessary work, maximize cache hits, and run independent tasks concurrently—turning slow build pipelines into faster sources of developer feedback.

Read original

Build Systems TypeScript Python Caching+4

airbnbAug 13, 20254 min readCurated summary

Migrating Airbnb’s JVM Monorepo to Bazel

Airbnb migrated its tens-of-millions-of-lines JVM monorepo from Gradle to Bazel over 4.5 years, achieving faster builds, testing, IntelliJ syncs, and development deployments. The move was driven by Bazel’s scalable remote execution, hermetic builds, and ability to provide shared infrastructure across Airbnb’s language-specific repositories. A gradual rollout, extensive automation, and close collaboration with service teams were central to making the migration successful. ## Results of the Migration - Build CSAT increased from 38% to 68%. - Local build and test times became 3–5 times faster. - IntelliJ syncs became 2–3 times faster. - Development-environment deployments became 2–3 times faster. ## Why Airbnb Chose Bazel ### Faster Builds Through Remote Execution - Large Gradle builds frequently took more than 20 minutes locally, while pre-merge CI builds had a p90 of 35 minutes. - Gradle had already been optimized with powerful machines and build sharding, but sharding caused underutilization and duplicated shared work. - Bazel’s cacheable actions and remote build execution enabled thousands of actions to run in parallel on short-lived workers. - “Build without the Bytes” reduced the amount of build output developers needed to download. - Bazel analysis runs in parallel, unlike the often single-threaded configuration phase of large Gradle projects. - Remote execution also improved local build performance, not just CI performance. ### More Reliable and Reproducible Builds - Gradle tasks could access the entire filesystem, creating accidental dependencies and race conditions. - Bazel sandboxes expose only declared inputs to each action, preventing undeclared files from affecting builds. - Bazel’s remote execution runs actions in identical containers with strict resource limits. - Using remote execution for both local and CI builds reduced differences between developer and CI environments. ### A Shared Build Infrastructure Layer Because Airbnb’s web, iOS, Python, Go, and JVM repositories all use Bazel, the company could standardize infrastructure for: - Remote caching - Remote build execution - Affected-target calculation - Build Event Protocol instrumentation and logging ## Starting with a Proof of Concept - Airbnb first migrated Viaduct, a large GraphQL monolith platform. - Viaduct was selected because it was complex, had slow builds, affected roughly 300 product engineers monthly, and had an infrastructure team willing to collaborate. - Bazel and Gradle initially coexisted, allowing developers to choose either system. - The team ported Viaduct’s build logic and created an automated Bazel build-file generator because the Gradle dependency graph continued to change. - Although Bazel was initially 2–4 times faster locally, developers did not adopt it immediately. - The team spent several additional months fixing missing integrations and bugs before Viaduct engineers voluntarily switched. ## Scaling Across the JVM Monorepo - Airbnb expanded breadth-first, aiming to make the entire repository compile and test under Bazel. - Gradle and Bazel continued to coexist during the migration. - This allowed developers to use Bazel locally while deployments still relied on Gradle. - Gradle provided a fallback when Bazel infrastructure, such as remote caching or execution, experienced incidents. - Maintaining two build graphs was costly, so Airbnb invested heavily in automation rather than requiring developers to maintain Bazel files manually. ## Automated Build-File Generation - The generator was inspired by Gazelle but was built internally to meet stricter performance requirements and handle dependency cycles. - It parses Java, Kotlin, and Scala source files to identify packages, imports, and symbol declarations. - These relationships are used to construct a file-level dependency graph. - Since generation ran on every commit before merging, Airbnb added external caching to keep it fast. - CI publishes a cached repository index for each mainline commit, allowing the generator to rescan only directories changed since that commit. Airbnb’s experience suggests that a large build-system migration is most effective when introduced incrementally: prove the benefits on a representative service, automate maintenance, preserve a fallback during rollout, and address developer workflow issues before expanding across the organization.

Read original

Build Systems Kotlin Java Monorepo+4