Build better software to build software better
Slack’s backend build pipeline for Quip and Slack Canvas once took 60 minutes, delaying feedback and slowing delivery. The team improved build performance by applying familiar software-engineering techniques—caching, parallelization, precise interfaces, and careful decomposition—using Bazel. The central argument is that build systems should be designed like high-performance programs: do less work, distribute unavoidable work, and define work units rigorously. ## Modeling Builds as Dependency Graphs - Applications can be represented as directed acyclic graphs of source files, intermediate artifacts, and deployable outputs. - A backend artifact depends on Python files, while a frontend artifact depends on TypeScript files. - Changing a Python file should rebuild the backend but not unrelated frontend components. - Clearly defined graph nodes allow build systems to optimize work rather than rebuilding everything. ## Caching and Hermetic Work - Caching avoids repeating expensive operations by storing outputs for known inputs. - The article uses a cached recursive `factorial()` function as an analogy: - The input is the cache key. - The return value is the cached artifact. - Effective caching requires work to be: - **Hermetic:** dependent only on explicitly provided inputs. - **Idempotent:** producing the same output for the same inputs. - Cache hit rate matters: poorly defined work units produce more cache misses. ## Granular Cache Units - Caching an entire `process_images(images, transforms)` operation is inefficient because changing one image invalidates the result for every image. - A more granular design caches `process_image(image, transform)` independently. - The higher-level operation can then reuse cached results and process only new image-transform combinations. - Smaller, well-defined units generally improve cache reuse and reduce rebuild time. ## Parallelizing Independent Work - Image processing can also be distributed across CPU threads using `ThreadPoolExecutor`. - Parallel work requires: - Completely specified inputs and outputs. - The ability to transfer data across thread, process, or network boundaries. - Handling completion and failure in any order. - APIs must document ordering guarantees; the threaded example returns images in completion order rather than input order. - Work-unit granularity affects scalability: - Too few large tasks limit available parallelism. - Too many tiny tasks may introduce coordination overhead. - The appropriate balance depends on the workload. ## Applying These Principles to Bazel - Bazel represents builds as directed acyclic graphs made of targets. - Each target defines: - Its input or dependency files. - Its output files. - The commands that transform inputs into outputs. - This structure provides the foundation for caching and parallel execution, just as explicit function inputs and outputs enable those optimizations in application code. The practical recommendation is to design build steps as small, hermetic, idempotent, and independently executable units. Combined with Bazel’s dependency graph, this lets teams avoid unnecessary work, maximize cache hits, and run independent tasks concurrently—turning slow build pipelines into faster sources of developer feedback.
Read original(opens in new tab)