How we migrated a live routing system using AI-assisted refactoring (opens in new tab)

Stream Router evolved from a small configuration file into a critical control-plane service routing Datadog’s massive metrics workload. Its original FoundationDB key-value model eventually hit transaction-size and performance limits because relational relationships were reconstructed in application code. Datadog redesigned the system around PostgreSQL and DuckDB, using AI-assisted, test-driven refactoring to accelerate the migration without disrupting production traffic.

Stream Router’s Role in Datadog’s Metrics Pipeline

• Datadog processes more than a hundred trillion events per day.
• Stream Router determines which Kafka cluster, topic, partitions, and sharding strategy should handle each datapoint.
• It serves both producers and queriers but does not process Kafka messages itself.
• Routing decisions change frequently as infrastructure evolves, making correctness and historical tracking essential.

From Configuration File to Control Plane

• In 2016, routing was managed through a small configuration file distributed to services.
• As the platform grew, the file expanded to thousands of lines and required manual edits and rollouts.
• Stream Router replaced this workflow with:
  • A centralized gRPC service
  • API-managed routes
  • Automated, gradual rollouts
• The write path used FoundationDB, while the read path served static RocksDB snapshots restored into memory.
• This eventually became a bottleneck as routing tables and operational changes grew larger.

Why the Key-Value Model Stopped Scaling

• Routes reference streams and sharding strategies, while rules reference routes.
• These relationships are inherently relational and require cross-entity validation.
• The KV implementation loaded tens of thousands of records into application processes and reconstructed database-like relationships in code.
• Some operations exceeded FoundationDB transaction-size limits.
• Moving to PostgreSQL without changing the access patterns would not solve the issue; certain operations were estimated to require 45 minutes because of thousands of sequential database round trips.
• The fundamental problem was the data model and application logic, not simply the choice of database.

Designing the New Storage Architecture

• The team redesigned the schema manually before using AI tools.
• The relational model introduced explicit foreign keys between:
  • Streams
  • Sharding strategies
  • Routes
  • Rules
• PostgreSQL was selected for the write path because it provided the required relational semantics and transaction model.
• DuckDB was selected for the read path because:
  • It is embeddable and suitable for snapshot-based serving
  • It supports array columns
  • Its SQL dialect is closely compatible with PostgreSQL
• Shared query logic could therefore work across both storage engines.

AI-Assisted Refactoring

• Claude and Cursor were used to accelerate a systematic, test-driven migration.
• For each method, developers supplied:
  • The old implementation
  • The new schema
  • A failing test
• AI generated an initial implementation, while tests determined whether it was correct.
• The models assisted with method-level refactoring rather than autonomously designing the architecture.
• Human expertise remained central to schema design, migration strategy, and evaluating system-level risks.

Foundations for a Safe Migration

• The migration benefited from infrastructure already present at Datadog.
• Stream Router’s storage layer was isolated behind an internal Controller interface.
• This modularity helped contain storage changes and enabled incremental refactoring.
• Existing tests and clear boundaries provided confidence in generated implementations while production traffic continued.

The central lesson is that AI was most effective as an accelerator inside a disciplined engineering process. A well-designed relational schema, modular storage abstraction, and failing tests provided the safety mechanisms; AI helped implement the resulting changes faster, but did not replace human architectural judgment.