Towards a science of scaling agent systems: When and why agent systems work
AI agent systems do not improve simply by adding more agents. Google Research’s evaluation of 180 configurations found that coordination helps substantially on parallelizable tasks but can severely hurt sequential workflows and tool-heavy tasks. The study proposes measurable design principles and a predictive model that selected the best architecture for 87% of unseen tasks. ## Defining Agentic Tasks The study distinguishes agentic tasks from static benchmarks by requiring: - Sustained, multi-step interaction with an external environment. - Iterative information gathering under partial observability. - Adaptive strategy changes based on environmental feedback. Researchers tested five architectures across Finance-Agent, BrowseComp-Plus, PlanCraft, and Workbench: - **Single-agent:** One agent handles reasoning and actions sequentially. - **Independent:** Agents work in parallel without communication and combine results at the end. - **Centralized:** An orchestrator delegates work and synthesizes outputs. - **Decentralized:** Agents communicate directly in a peer-to-peer network. - **Hybrid:** Hierarchical oversight is combined with peer coordination. ## Coordination Must Match the Task - Multi-agent systems produced very different results across GPT, Gemini, and Claude models. - On parallelizable financial reasoning tasks, centralized coordination improved performance by **80.9%** over a single agent. - Parallel agents can independently analyze areas such as revenue, costs, and market comparisons before combining their findings. - On sequential planning tasks, every multi-agent architecture performed worse, with declines of **39–70%**. - Communication and synchronization overhead can fragment reasoning and consume the available cognitive budget. ## The Tool-Coordination Trade-off - As tasks require more tools, coordinating multiple agents becomes increasingly expensive. - Tool-heavy systems, such as coding agents with access to 16 or more tools, face a disproportionate coordination “tax.” - Adding agents is therefore especially risky when actions must be tightly ordered or frequently synchronized. ## Architecture and Reliability - Architecture affects not only performance but also how errors spread. - Independent agents amplified errors by up to **17.2×**, because no mechanism checked their intermediate results. - Centralized systems limited error amplification to **4.4×**. - An orchestrator acts as a validation bottleneck, detecting and containing mistakes before they propagate. ## Predicting the Best Architecture - The researchers built a predictive model using properties such as task decomposability and tool count. - The model achieved an **R² of 0.513**. - It correctly predicted the optimal coordination strategy for **87% of unseen task configurations**. - These results point toward systematic, task-driven agent design rather than relying on the assumption that more agents are always better. For practical deployments, choose architecture based on the task: use coordinated parallel agents for decomposable work, simpler sequential systems for tightly ordered reasoning, and centralized oversight when reliability and error containment are priorities.
Read original(opens in new tab)
