How we migrated our static analyzer from Java to Rust

Datadog migrated its static analyzer from Java to Rust after finding that ANTLR-based parsing was too slow and language support was incomplete. Rust’s strong integration with Tree-sitter enabled broader language coverage, faster scans, and lower memory usage. The migration preserved behavioral parity while tripling performance and reducing memory consumption tenfold.

Why Performance Became a Priority

• Datadog runs analysis directly in customers’ CI environments, often on resource-constrained runners.
• On a two-core, 7 GB GitHub Actions runner, medium repositories took about five minutes to scan instead of the target of under three minutes.
• Codiga’s previous hosted environment used large, tuned servers, which masked some performance problems.
• Java also required customers to use JVM 17+, potentially conflicting with JVM versions already installed in their CI environments.
• Improving Java offered limited upside, so the team considered a rewrite despite its cost and risk.

Static Analyzer Architecture

• The analyzer consists primarily of:
  • A parsing layer that builds an abstract syntax tree (AST).
  • An execution layer that analyzes the AST, reports violations, and offers fixes.
• Tree-sitter generates the AST.
  • The existing Java binding lacked important functionality, including Tree-sitter pattern matching.
  • Tree-sitter’s core libraries are implemented in Rust, where support was more complete.
• Analysis rules are written in JavaScript and were originally executed through GraalVM’s polyglot capabilities.
• Fast parsing, pattern matching, and rule execution were central to meeting the desired CI performance.

Migrating from Java to Rust

• Rust was selected because it is a first-class part of the Tree-sitter ecosystem and provided better access to its features.
• The migration required:
  • Feature parity with the Java implementation.
  • Identical analysis results and reported violations.
  • No execution-time regressions.
• Migrating the parser was relatively straightforward because Rust support came directly from Tree-sitter.
• The Rust implementation:
  • Tripled analyzer performance.
  • Reduced memory usage by a factor of ten.
• JavaScript execution moved from GraalVM to deno-core, a Rust-based V8 integration.
  • Only the core JavaScript functionality was included.
  • Disk and network capabilities were excluded because analysis rules do not need them, improving security.

Migration Strategy and Rust Adoption

• The team treated automated equivalence and performance tests as requirements for a successful rewrite.
• Rust allowed the analyzer to integrate more directly with its key dependencies rather than maintaining a separate Java binding.
• The broader migration also required replacing supporting Java components with corresponding Rust libraries; the article indicates that these mappings were documented as part of the transition.

Overall, the move to Rust was justified by the analyzer’s deployment model: faster execution and lower resource consumption directly improved the experience of customers running scans in constrained CI environments.