Detecting malicious pull requests at scale with LLMs

Datadog built an LLM-powered security reviewer, BewAIre, to examine every pull request for malicious intent as AI-assisted development increases code volume and reviewer fatigue. Unlike traditional static analysis, the system evaluates the design and likely purpose of changes, not just known syntactic patterns. In testing, it achieved over 99.3% accuracy with a 0.03% false-positive rate and now operates in production.

Why Conventional Review Is Struggling

• Datadog processes nearly 10,000 pull requests per week across internal and external repositories, with volume growing as coding assistants become commonplace.
• More changes increase the attack surface and make subtle attacks harder to notice.
• Reviewers, linters, and scanners can be overwhelmed by verbose or repetitive AI-generated code.
• Static analysis is effective at identifying known bad patterns but generally lacks understanding of developer intent.
• Legitimate security-sensitive changes, such as permission-model updates, can resemble malicious behavior.

The tj-actions/changed-files Attack

• An attacker used a compromised personal access token belonging to tj-actions-bot to modify index.js.
• The injected Python code extracted secrets and other sensitive information from a GitHub Actions runner’s process memory.
• Base64 encoding concealed both the malicious code reference and output written to build logs.
• The commit was made to look like a legitimate renovate[bot] update.
• Version tags were redirected to the malicious commit, allowing the compromise to spread quickly.
• Even protected branches, two-person reviews, and other standard controls could miss an attack disguised as a routine dependency or maintenance change.

BewAIre’s LLM-Based Review Pipeline

• Ingestion: Collects pull requests merged into the default branch across Datadog repositories.
• Preprocessing: Normalizes each pull request, extracts its diff, and adds metadata such as the author and repository type.
• Inference: Sends the code changes and contextual information to an LLM to reason about intent and classify the change as benign or malicious.
• Security signals: Converts each verdict into a Datadog security signal that appears in internal dashboards and can trigger alerts or pages for security engineers.
• Each finding includes a written explanation of why the change may be malicious.

Improving Accuracy and Limiting Noise

• Datadog used prompt engineering and data tuning to improve the model’s ability to distinguish malicious behavior from legitimate changes.
• Suppression rules were added for known-safe patterns.
• Testing on a curated, balanced dataset of hundreds of malicious and benign pull requests produced:
  • More than 99.3% overall accuracy.
  • More than 93.5% balanced accuracy.
  • A 0.03% false-positive rate.
• The team also designed the system to address model degradation caused by context limits and continuously evaluated it against real-world attacks, including the tj-actions and Nx incidents.

Production Deployment

• BewAIre reviews Datadog pull requests in real time and is already deployed across the company’s repositories.
• The system is available in Preview to Static Code Analysis customers.
• Its purpose is to add scalable, intent-focused detection without imposing stricter manual review requirements that could slow development.

Datadog’s approach suggests that LLMs can complement—not replace—static analysis and human investigation by providing continuous, intent-aware security review at large scale.