gitlab·Aug 12, 2026·2 min readCurated summary

How GitLab tracks vulnerabilities through refactors and reformatting

GitLab improved vulnerability tracking to prevent duplicate findings after harmless code changes such as added comments, blank lines, or reformatting. Its normalized Scope+Offset fingerprint ignores non-functional lines while still identifying vulnerabilities by their enclosing scope and relative location. In testing, the method eliminated duplicates in a worst-case benchmark and is now available as `scope_offset_compressed`. ## The Problem with Line-Based Tracking - Vulnerabilities can appear to be new when developers: - Add comments or blank lines - Reformat files - Move functions or other code - Duplicate findings force security teams to repeat triage and reduce confidence in scan results. - GitLab’s 2022 Scope+Offset method improved on line-based tracking by using: - The narrowest enclosing scope, such as a module, class, or function - The finding’s line offset within that scope - This reduced unnecessary re-auditing by about 30%, but comments and blank lines could still change the offset. ## Normalized Scope+Offset Fingerprinting - The improved method ignores comments and blank lines when calculating a vulnerability’s fingerprint. - Because these lines do not change program behavior, they no longer change the vulnerability’s identity. - The approach preserves the previous method’s precision and reuses the scanner’s existing parse tree, so scan times are unaffected. - It does not require changes to the security report format and works with heterogeneous SAST toolchains. ## Benchmark Results - Researchers tested 439 source files across: - C/C++, C#, Go, Java, JavaScript, Python, and Ruby - They generated 2,247 commits, each adding a comment or blank line immediately before a known vulnerability. - The original method produced 1,361 duplicate fingerprints, representing a 77% increase over baseline. - The normalized method produced zero duplicates and reduced unique fingerprints by 43%. ## GitLab Availability - The algorithm is available in GitLab as: - `scope_offset_compressed` - It supports C#, C/C++, Go, Java, JavaScript, Python, Ruby, and PHP. - The accompanying research paper, *Vulnerability Tracking using Normalized Scope+Offset*, is scheduled for presentation at the ASE 2026 Industry Showcase. For teams using GitLab security scanning, `scope_offset_compressed` provides more stable vulnerability tracking without slowing scans or changing existing report integrations.

Read original(opens in new tab)

Static AnalysisGitlabDevSecOpsSast+3

gitlab·Jul 16, 2026·2 min readCurated summary

GitLab Duo Security Review spots logic flaws scanners miss

GitLab’s Duo Security Review uses AI-driven reasoning to identify authorization gaps, data exposure, business-logic errors, and race conditions that pattern-based scanners often miss. It reviews merge requests in context, validates findings to reduce false positives, and reports issues before they reach production. The feature complements—not replaces—automated scanners and human security review. ## Where Traditional Scanners Fall Short - Static scanners detect recognizable patterns such as unsanitized inputs, secrets, and unsafe deserialization. - Logic flaws depend on application-specific context, including: - Broken object- or function-level authorization - Sensitive data exposed through otherwise normal serialization - Incorrect workflow order, manipulated prices, checkout bypasses, and race conditions - Manual reviews, penetration tests, and bug bounties are difficult to scale or may happen too late. ## AI-Powered Review for Merge Requests - Security Review Flow analyzes the diff alongside original files, related code, and MR discussion. - It can detect missing authorization, information disclosure, mass assignment, business-logic flaws, and stateful workflow races. - An independent validation pass reviews findings to filter likely false positives. - GitLab’s application security team has used it on internal merge requests. ## Findings and Review Workflow - Findings appear as diff threads and an internal summary; public projects keep security details in the internal note. - Each finding includes: - Vulnerability type and CWE reference - Severity from critical to low - Tier: exploitable, logic flaw, or design issue - Explanation and suggested remediation - Critical and high findings request changes; medium and low findings generate comments. - The tool never approves an MR, leaving the final decision to a human. - Developers can discuss findings, apply suggested fixes, dismiss false positives, accept risks, and request another review after changes. ## Availability and Cost - The feature is in public beta for GitLab Ultimate customers on GitLab.com, Self-Managed, and Dedicated. - Access is available through a Duo Agent Platform trial or included GitLab Credits. - Pricing depends on diff complexity and model selection, so GitLab recommends testing it on a limited number of MRs first. Organizations should use Security Review Flow as an additional layer alongside scanners and expert review, applying it early in the merge-request process when security fixes are least expensive.

Read original(opens in new tab)

Static AnalysisDevSecOpsAi SecurityGitlab Duo+4

dropbox·Jun 12, 2026·3 min readCurated summary

How Dropbox uses MCP and Dash to close the design-to-code security gap

Dropbox found a significant gap between security design reviews and implementation. Only 12% of implementing pull requests linked back to their original threat models, and the median delay between review and code submission was about five weeks. To close this gap, Dropbox built a system using Dash, Model Context Protocol (MCP), and foundational models to automatically retrieve relevant security requirements and compare them with code during review. ## The Design-to-Code Gap - Threat models document risks, attack scenarios, and agreed-upon mitigations during security review. - These documents often remain in wikis or documentation systems, while implementation happens later through pull requests. - At Dropbox: - Only 12% of implementing PRs linked to their original design review. - Among 79 verified pairs, 54% of PRs were opened more than a month after the review. - The median delay was approximately five weeks, with some delays exceeding 11 months. - Only 29% of PRs were opened within two weeks of the security review. - About 15% of design reviews were filed retroactively, suggesting that some security-sensitive work was not identified early enough. ## Why Existing Tools Fall Short - Static analysis can detect whether certain security patterns or controls exist. - It generally cannot determine whether those controls satisfy the specific requirements agreed upon during design review. - Manual linking between PRs and threat models depends on engineers remembering additional workflow steps. - Reminder bots may improve compliance temporarily, but adherence tends to decline. - The core problem is not a lack of security documentation; it is the difficulty of making that existing context available during implementation. ## Dash and MCP as a Context Bridge - Dash already indexes Dropbox content and connected applications, including threat models and engineering documentation. - Dropbox used Dash’s MCP server to let an AI security agent search and read this content. - MCP provides a common interface for bringing multiple context sources into one agent session. - When a PR is opened, the agent retrieves relevant threat models and supporting documents. - A foundational model then compares the documented requirements with the proposed code. - For example, it can identify whether code implementing an endpoint enforces authentication required by the corresponding threat model. - Unlike traditional static analysis, this approach evaluates code against documented security intent, not just known code patterns. ## Integrating Security into Code Review - Dropbox surfaced the system directly within the existing code-review workflow rather than creating a separate security process. - This allows developers and reviewers to receive relevant security context where implementation decisions are already evaluated. - The same mechanism can potentially identify missing reviews when code appears security-sensitive but has no corresponding threat model. Dropbox’s approach shows how retrieval and AI reasoning can reconnect design decisions with implementation. Organizations can apply the same pattern beyond security—for example, to compliance requirements, privacy reviews, accessibility standards, or other design-to-code checks.

Read original(opens in new tab)

Static AnalysisLarge Language ModelsModel Context ProtocolCode Review+2

spotify·May 18, 2026·3 min readCurated summary

Coding Is No Longer the Constraint: Scaling Developer Experience to Teams and Agents at Spotify | Spotify Engineering

Spotify argues that AI has shifted software development’s main constraint from writing code to coordinating people, systems, and decisions. Years of investment in standardized platforms, automation, and developer experience enabled Spotify to adopt AI coding tools at extraordinary scale. The company’s experience suggests that consistent infrastructure and strong feedback loops are essential for making both human developers and coding agents effective. ## Rapid AI Adoption - More than 99% of Spotify engineers use AI coding tools weekly. - 94% report improved productivity. - Pull request frequency has increased by 76%, with most PRs created by developers working alongside AI agents. - Adoption accelerated sharply after the release of Claude Opus 4.5. ## Fleet Management Before AI Agents - Spotify’s codebase was growing seven times faster than its engineering workforce. - Developers increasingly spent time on dependency upgrades, API migrations, and vulnerability fixes. - Fleet Management automated changes across hundreds or thousands of components. - Its orchestration system, Fleetshift, has merged more than 2.5 million maintenance PRs, most without human intervention. - This approach reduced migrations from work taking weeks or months across many teams to centrally managed operations. ## Honk: A Background Coding Agent - Deterministic scripts struggled with complex refactoring and the edge cases found across large codebases. - Spotify created Honk, a background coding agent powered by Claude through the Agent SDK. - Honk runs in Kubernetes pods, allowing many coding sessions to execute concurrently. - It can use trusted tools and run builds in CI across multiple operating systems. - Fleetshift identifies targets, schedules work, and tracks PRs, while Honk performs the code changes. - A recent Java migration across Spotify’s backend services took three days. - Engineers can invoke Honk through Slack, where it uses conversation context to create and return PRs. - Honk v2 adds shared sessions, team projects, and agent orchestration through Chirp. ## Standardization Improves Agent Performance - Spotify’s principle of limiting the number of technologies it supports reduces decisions and improves collaboration. - Consistent service architectures and design patterns also give AI agents better reference material. - Agents perform worse in fragmented codebases with inconsistent conventions. - Backstage provides a unified internal developer portal and catalog for software components. - Spotify exposes Backstage capabilities to agents through MCP integrations and command-line tools. - Agents can discover component ownership, read documentation, and contact responsible teams. ## Guardrails Through Backstage - Backstage’s Soundcheck and “golden state” define recommended technologies and practices. - Teams can assess their components against these standards. - Static analysis and linting provide immediate feedback when developers or agents use unsuitable patterns. - This creates a feedback loop that helps agents correct their work and drives consistency across the organization. Spotify’s experience indicates that scaling AI development requires more than giving engineers access to models. Organizations should invest in standardized platforms, searchable component metadata, automated fleet-wide workflows, and strong validation systems so agents can operate reliably at team scale.

Read original(opens in new tab)

Static AnalysisAI AgentsKubernetesCI/CD+4

github·May 15, 2026·3 min readCurated summary

Raising the bar: Quality, shared responsibility, and the future of GitHub’s bug bounty program

GitHub is reaffirming its commitment to external security researchers while tightening bug bounty submission standards. Rising report volumes—partly driven by AI and other tools—have increased both valuable findings and unvalidated noise. GitHub’s central message is that tools are welcome, but researchers remain responsible for validating vulnerabilities, demonstrating impact, and understanding the platform’s shared security boundaries. ## Rising Submission Volume - New tools, including AI, have lowered the barrier to security research and expanded the number of people examining attack surfaces. - GitHub has also seen more reports that: - Lack a working proof of concept - Describe only theoretical attack scenarios - Concern categories already listed as ineligible - Because this challenge affects the wider industry, some bug bounty programs have shut down; GitHub instead plans to improve its program. ## Requirements for Strong Reports - Submissions must include a working proof of concept demonstrating concrete security impact. - Researchers should show what an attacker can actually accomplish, rather than merely describing a possible attack path. - Reports must respect GitHub’s published scope and ineligible findings list. Examples of generally ineligible issues include: - DMARC, SPF, or DKIM configuration problems - User enumeration - Missing security headers without a demonstrated attack path - Scanner, static-analysis, or AI-generated findings must be manually validated before submission. - Unverified false positives create unnecessary triage work and may affect a researcher’s HackerOne Signal and reputation. ## AI Is Welcome, but Validation Is Required - GitHub supports the use of AI in security research and uses AI internally. - AI-assisted reports are acceptable when findings are reproduced, verified, and supported by a working proof of concept. - Researchers remain accountable for the accuracy of their submissions, regardless of which tools produced them. - GitHub recommends a concise report structure: - A short issue summary - Clear reproduction steps and evidence, such as screenshots, HTTP requests, or terminal output - An impact statement explaining what an attacker can achieve - Lengthy theoretical explanations and AI-generated filler can obscure the actual vulnerability and slow triage. ## Shared Responsibility and GitHub’s Security Boundary - GitHub protects users through automated scanning, manual review, and other systems for detecting malicious content. - Users are still responsible for deciding what repositories, issues, code, and scripts to trust. - Users should review content before executing or interacting with it. - Cloning a repository is considered an act of trust because Git hooks, build scripts, and other automation may run locally. - Users must also secure their own environments, including tokens, credentials, and local security settings. - Scenarios generally do not bypass GitHub’s security controls when they require victims to deliberately engage with attacker-controlled content. ## Common Shared-Responsibility Scenarios - Prompt injection in content a user intentionally provides to an AI tool - Git hooks or filters executing code from a repository the user checked out - Malicious content in a repository the user chose to clone - Unexpected LLM output caused by untrusted input supplied by the user Research into these areas remains useful when it identifies a way to bypass an actual GitHub security control without requiring the user to actively trust malicious content.

Read original(opens in new tab)

Static AnalysisGithubGitAi Security+4

netflix·May 8, 2026·4 min readCurated summary

Scaling ArchUnit with Nebula ArchRules

Netflix’s Nebula ArchRules extends ArchUnit so architectural and API-lifecycle rules can be shared across thousands of Gradle repositories. Unlike AST-based tools, ArchUnit analyzes compiled JVM bytecode, supports multiple JVM languages, and offers a type-safe Java API for authoring and testing rules. The approach helps identify unsafe API usage, technical debt, and deviations from Netflix’s preferred development practices at fleet scale. ## The API Lifecycle Problem - Netflix operates tens of thousands of Java repositories in a polyrepo environment. - A library incident involving a backwards-incompatible change highlighted the difficulty of deciding when deprecated APIs can safely be removed. - Netflix introduced lifecycle annotations: - `@Deprecated` for APIs scheduled for removal - `@Public` for APIs intended for downstream use - `@Experimental` for APIs that may change - Unannotated APIs are treated as internal - The remaining challenge was identifying downstream projects that use internal, experimental, or deprecated APIs incorrectly. - The same tooling could support large migrations, such as major Spring Boot upgrades. ## Why ArchUnit - ArchUnit is an open-source library commonly used within JUnit suites to enforce architectural rules. - It is built on ASM and analyzes compiled JVM bytecode rather than source syntax. - Its main strengths are: - Cross-language JVM support for Java, Kotlin, Scala, and other JVM languages - A fluent builder API for readable rule definitions - A lower-level API for complex custom analysis - Access to class relationships, dependencies, and call sites through its class graph - Standard ArchUnit is primarily designed for one repository, so Netflix created Nebula ArchRules to distribute rules across many Gradle projects. ## Bytecode Analysis vs. AST Analysis - AST-based tools such as PMD inspect source-code structure and can be sensitive to language-specific syntax and syntactic sugar. - Supporting multiple JVM languages may require separate rules for each language. - ASM analyzes the bytecode that will actually execute, regardless of how the source was written. - This makes rules more consistent across Java, Kotlin, Scala, and other JVM languages. ## Rule Authoring - Tools such as PMD and SpotBugs are generally optimized for built-in rules or third-party plugins rather than custom rule development. - PMD custom rules may require difficult-to-maintain XPath expressions and separate tooling for testing. - ArchUnit rules are written as type-safe, fluent Java code. - Rules can be unit tested directly by passing them class references, without running a separate analysis process. - ArchUnit’s class graph provides contextual information about dependencies and call relationships, enabling more sophisticated checks. ## ArchRules Libraries - The Nebula ArchRules Library Plugin adds an `archRules` source set to a Gradle project. - A class implementing `ArchRulesService` exposes a `Map<String, ArchRule>`: - The map key names the rule. - The `ArchRule` defines the constraint using ArchUnit’s API. - Rule code and its dependencies are kept separate from the application’s main code. - Gradle publishes the rules in a separate JAR using the `arch-rules` classifier and an `arch-rules` usage attribute. - Downstream projects must use Gradle Module Metadata to resolve the rules variant. ## Standalone and Bundled Rule Libraries - Standalone rule libraries contain only `archRules` code. - They are useful for: - Enforcing rules around APIs the organization does not own - Checking usage of Java or open-source libraries - Applying generic rules, such as prohibiting use of deprecated APIs - Bundled rule libraries contain both normal library code and rules specific to how that library should be used. - Netflix maintains open-source standalone rule libraries as examples and reusable building blocks. Nebula ArchRules turns ArchUnit from a repository-local testing library into a reusable organization-wide policy mechanism. Teams can publish rules as Gradle artifacts and apply them consistently across JVM projects, making API governance, dependency policies, and architectural standards easier to enforce at scale.

Read original(opens in new tab)

Static AnalysisKotlinJavaJvm+4

github·Apr 14, 2026·2 min readCurated summary

How exposed is your code? Find out in minutes—for free

GitHub’s Code Security Risk Assessment offers organizations a free, one-click way to uncover vulnerabilities that manual reviews and limited scanning tools may miss. Using CodeQL, it scans up to 20 active repositories and provides a prioritized view of risks by severity, language, repository, and vulnerability type. The assessment also connects findings to GitHub Code Security and Copilot Autofix, helping teams move from discovery to remediation. ## What the Assessment Reveals - Scans up to 20 of an organization’s most active repositories. - Reports vulnerabilities by severity: - Critical - High - Medium - Low - Breaks findings down by programming language. - Identifies the specific security rules and vulnerability classes detected. - Highlights the repositories with the greatest number of vulnerabilities. - Shows which findings are eligible for Copilot Autofix. - Uses CodeQL, GitHub’s static analysis engine. The assessment is free for organization administrators and security managers using GitHub Enterprise Cloud or GitHub Team. Scanning does not consume the organization’s GitHub Actions minutes quota. ## Combining Code and Secret Exposure The Code Security Risk Assessment complements GitHub’s existing Secret Risk Assessment. - Both assessments can be launched from one entry point. - A tabbed interface separates secret exposure findings from code vulnerabilities. - The combined view helps organizations understand their overall security posture. - Secret Protection focuses on preventing credential leaks. - Code Security identifies and helps remediate vulnerabilities in source code. GitHub cites 2025 usage figures for Secret Protection, including nearly 2 billion scanned pushes and 19 million blocked secret exposures. ## From Detection to Remediation Finding vulnerabilities is only the first step; reducing risk requires fixing them. - GitHub reports that Copilot Autofix resolved 460,258 security alerts in 2025. - Half of vulnerability alerts were fixed directly in pull requests. - Mean remediation time was 0.66 hours with Copilot Autofix, compared with 1.29 hours for manual fixes. - Assessment results show how many discovered vulnerabilities could potentially be fixed automatically. - Organizations can enable GitHub Code Security directly from the results page. The assessment is intended for organizations with no current scanning, teams evaluating existing tools, or security leaders seeking broader visibility across repositories. Running it provides a quick, no-cost baseline for prioritizing security work.

Read original(opens in new tab)

Static AnalysisGitHub ActionsSecret ScanningCodeql+3

github·Mar 23, 2026·2 min readCurated summary

GitHub expands application security coverage with AI‑powered detections

GitHub is expanding application security coverage with AI-powered detections that complement CodeQL’s traditional static analysis. The approach targets languages and frameworks that are difficult to support through semantic analysis alone, including Bash, Dockerfiles, Terraform, and PHP. Planned for public preview in early Q2, the system brings detection, automated remediation, and enforcement directly into pull requests. ## Hybrid Static Analysis and AI Detection - CodeQL remains the primary tool for deep analysis of supported languages. - AI-powered detections extend coverage to scripts, infrastructure definitions, and less-supported ecosystems. - The system can identify vulnerabilities and suggest fixes within the pull request workflow. - Internal testing analyzed more than 170,000 findings in 30 days, receiving positive feedback from over 80% of developers. - Early supported areas include: - Shell/Bash - Dockerfiles - Terraform/HCL - PHP - The capability is part of GitHub’s broader agentic detection platform, which also supports code quality and code review. ## Security Findings in Pull Requests - GitHub automatically analyzes changes when a pull request is opened. - It selects CodeQL or AI-powered detection based on the code being reviewed. - Findings appear alongside existing code-scanning results, without requiring developers to switch tools. - Example risks include: - Unsafe string-built SQL queries or commands - Weak cryptographic algorithms - Infrastructure configurations exposing sensitive resources - Detecting issues during review allows teams to address vulnerabilities before code is merged or deployed. ## Copilot Autofix for Remediation - GitHub connects detection with Copilot Autofix, which proposes fixes developers can review, test, and apply. - Autofix resolved more than 460,000 security alerts in 2025. - Alerts were resolved in an average of 0.66 hours with Autofix, compared with 1.29 hours without it. - This reduces the gap between discovering a vulnerability and correcting it. ## Security Enforcement at Merge - GitHub positions pull requests as the point where security policies can be enforced. - Detection, remediation, and governance operate within the same workflow. - Teams can reduce risk without adding separate post-deployment review steps. - GitHub plans to demonstrate the technology at RSAC, highlighting hybrid detection and developer-native remediation. GitHub’s recommendation is effectively to combine CodeQL’s precision with AI-based coverage for modern, diverse repositories, while using Copilot Autofix and merge policies to turn findings into timely, enforceable fixes.

Read original(opens in new tab)

Static AnalysisDocker & ContainersTerraformAi Security+4

toss·Dec 23, 2025Original article

Creating Always-Up-To (opens in new tab)

Managing complex multi-page onboarding funnels often leads to documentation that quickly becomes decoupled from the actual codebase, creating confusion for developers. To solve this, the Toss team developed an automated system that uses static code analysis to generate funnel flowcharts that are never outdated. By treating the source code as the "Source of Truth," they successfully transformed hard-to-track navigation logic into a synchronized, visual map. ### The Limitations of Manual Documentation * Manual diagrams fail to scale when a funnel contains high-frequency branching, such as the 82 distinct conditions found across 39 onboarding pages. * Traditional documentation becomes obsolete within days of a code change because developers rarely prioritize updating external diagrams during rapid feature iterations. * Complex conditional logic (e.g., branching based on whether a user is a representative or an agent) makes manual flowcharts cluttered and difficult to read. ### Static Analysis via AST * The team chose static analysis over runtime analysis to capture all possible navigation paths simultaneously without the need to execute every branch of the code. * They utilized the `ts-morph` library to parse TypeScript source code into an Abstract Syntax Tree (AST), which represents the code structure in a way the compiler understands. * This method allows for a comprehensive scan of the project to identify every instance of navigation calls like `router.push()` or `router.replace()`. ### Engineering the Navigation Edge Data Structure * A "Navigation Edge" data structure was designed to capture more than just the destination; it includes the navigation method, query parameters, and the exact line number in the source code. * The system records the "context" of a transition by traversing the AST upwards from a navigation call to find the parent `if` statements or ternary operators, effectively documenting the business logic behind the path. * By distinguishing between `push` (which adds to browser history) and `replace` (which does not), the documentation provides insights into the intended user experience and "back button" behavior. ### Tracking Hidden Navigation and Constants * **Custom Hook Analysis:** Since navigation logic is often abstracted into hooks, the tool scans `import` declarations to follow and analyze logic within external hook files. * **Constant Resolution:** Because developers use constants (e.g., `URLS.PAYMENT_METHOD`) rather than raw strings, the system parses the project's constant definition files to map these variables back to their actual URL paths. * **Source Attribution:** The system flags whether a transition originated directly from a page component or an internal hook, making it easier for developers to locate the source of a specific funnel behavior. ### Conclusion For teams managing complex user journeys, automating documentation through static analysis is a powerful way to eliminate technical debt and synchronization errors. By integrating this extraction logic into the development workflow, the codebase remains the definitive reference point while providing stakeholders with a clear, automated visual of the user experience.

Static AnalysisTypeScriptWeb DevelopmentAbstract Syntax Tree+3

datadog·Oct 21, 2025·3 min readCurated summary

Detecting malicious pull requests at scale with LLMs

Datadog built an LLM-powered security reviewer, BewAIre, to examine every pull request for malicious intent as AI-assisted development increases code volume and reviewer fatigue. Unlike traditional static analysis, the system evaluates the design and likely purpose of changes, not just known syntactic patterns. In testing, it achieved over 99.3% accuracy with a 0.03% false-positive rate and now operates in production. ## Why Conventional Review Is Struggling - Datadog processes nearly 10,000 pull requests per week across internal and external repositories, with volume growing as coding assistants become commonplace. - More changes increase the attack surface and make subtle attacks harder to notice. - Reviewers, linters, and scanners can be overwhelmed by verbose or repetitive AI-generated code. - Static analysis is effective at identifying known bad patterns but generally lacks understanding of developer intent. - Legitimate security-sensitive changes, such as permission-model updates, can resemble malicious behavior. ## The tj-actions/changed-files Attack - An attacker used a compromised personal access token belonging to `tj-actions-bot` to modify `index.js`. - The injected Python code extracted secrets and other sensitive information from a GitHub Actions runner’s process memory. - Base64 encoding concealed both the malicious code reference and output written to build logs. - The commit was made to look like a legitimate `renovate[bot]` update. - Version tags were redirected to the malicious commit, allowing the compromise to spread quickly. - Even protected branches, two-person reviews, and other standard controls could miss an attack disguised as a routine dependency or maintenance change. ## BewAIre’s LLM-Based Review Pipeline - **Ingestion:** Collects pull requests merged into the default branch across Datadog repositories. - **Preprocessing:** Normalizes each pull request, extracts its diff, and adds metadata such as the author and repository type. - **Inference:** Sends the code changes and contextual information to an LLM to reason about intent and classify the change as benign or malicious. - **Security signals:** Converts each verdict into a Datadog security signal that appears in internal dashboards and can trigger alerts or pages for security engineers. - Each finding includes a written explanation of why the change may be malicious. ## Improving Accuracy and Limiting Noise - Datadog used prompt engineering and data tuning to improve the model’s ability to distinguish malicious behavior from legitimate changes. - Suppression rules were added for known-safe patterns. - Testing on a curated, balanced dataset of hundreds of malicious and benign pull requests produced: - More than 99.3% overall accuracy. - More than 93.5% balanced accuracy. - A 0.03% false-positive rate. - The team also designed the system to address model degradation caused by context limits and continuously evaluated it against real-world attacks, including the tj-actions and Nx incidents. ## Production Deployment - BewAIre reviews Datadog pull requests in real time and is already deployed across the company’s repositories. - The system is available in Preview to Static Code Analysis customers. - Its purpose is to add scalable, intent-focused detection without imposing stricter manual review requirements that could slow development. Datadog’s approach suggests that LLMs can complement—not replace—static analysis and human investigation by providing continuous, intent-aware security review at large scale.

Read original(opens in new tab)

Static AnalysisLarge Language ModelsPrompt EngineeringCode Review+4

datadog·May 23, 2024·3 min readCurated summary

How we migrated our static analyzer from Java to Rust

Datadog migrated its static analyzer from Java to Rust after finding that ANTLR-based parsing was too slow and language support was incomplete. Rust’s strong integration with Tree-sitter enabled broader language coverage, faster scans, and lower memory usage. The migration preserved behavioral parity while tripling performance and reducing memory consumption tenfold. ## Why Performance Became a Priority - Datadog runs analysis directly in customers’ CI environments, often on resource-constrained runners. - On a two-core, 7 GB GitHub Actions runner, medium repositories took about five minutes to scan instead of the target of under three minutes. - Codiga’s previous hosted environment used large, tuned servers, which masked some performance problems. - Java also required customers to use JVM 17+, potentially conflicting with JVM versions already installed in their CI environments. - Improving Java offered limited upside, so the team considered a rewrite despite its cost and risk. ## Static Analyzer Architecture - The analyzer consists primarily of: - A parsing layer that builds an abstract syntax tree (AST). - An execution layer that analyzes the AST, reports violations, and offers fixes. - Tree-sitter generates the AST. - The existing Java binding lacked important functionality, including Tree-sitter pattern matching. - Tree-sitter’s core libraries are implemented in Rust, where support was more complete. - Analysis rules are written in JavaScript and were originally executed through GraalVM’s polyglot capabilities. - Fast parsing, pattern matching, and rule execution were central to meeting the desired CI performance. ## Migrating from Java to Rust - Rust was selected because it is a first-class part of the Tree-sitter ecosystem and provided better access to its features. - The migration required: - Feature parity with the Java implementation. - Identical analysis results and reported violations. - No execution-time regressions. - Migrating the parser was relatively straightforward because Rust support came directly from Tree-sitter. - The Rust implementation: - Tripled analyzer performance. - Reduced memory usage by a factor of ten. - JavaScript execution moved from GraalVM to `deno-core`, a Rust-based V8 integration. - Only the core JavaScript functionality was included. - Disk and network capabilities were excluded because analysis rules do not need them, improving security. ## Migration Strategy and Rust Adoption - The team treated automated equivalence and performance tests as requirements for a successful rewrite. - Rust allowed the analyzer to integrate more directly with its key dependencies rather than maintaining a separate Java binding. - The broader migration also required replacing supporting Java components with corresponding Rust libraries; the article indicates that these mappings were documented as part of the transition. Overall, the move to Rust was justified by the analyzer’s deployment model: faster execution and lower resource consumption directly improved the experience of customers running scans in constrained CI environments.

Read original(opens in new tab)

Static AnalysisDatabase DesignCI/CDJavaScript+4

figma·Apr 24, 2024·3 min readCurated summary

Speeding Up C++ Build Times | Figma Blog

Figma cut C++ build times roughly in half by addressing unnecessary header inclusion rather than relying solely on faster hardware or caching. The team found that compiled bytes were growing much faster than the codebase itself, making transitive header dependencies the main culprit. They combined automated include analysis with CI-based measurement to prevent both unused includes and costly dependency regressions. ## Why Build Times Were Getting Worse - In 2023, Figma’s codebase grew by about 10%, but build times increased by 50%. - C++ builds were a major productivity problem and a top concern in internal developer surveys. - Faster M1 Max machines, Ccache, and remote caching provided only temporary or insufficient improvements. - The team observed that build times were closely related to the amount of code passed to the compiler after preprocessing. ## How C++ Header Inclusion Affects Builds - The preprocessor expands every `#include` into a single large file before compilation. - Transitive dependencies are included as well: - If file C includes B, and B includes A, C receives the contents of both A and B. - As a result, a small source change can cause the compiler to process a very large amount of unrelated code. ## Removing Unnecessary Includes - Figma suspected that many files included headers they did not use directly or relied on headers only for transitive dependencies. - Removing unnecessary includes from the largest files produced: - A 31% reduction in compiled bytes. - A 25% reduction in cold build time. - These results confirmed that compiled byte volume was strongly correlated with build performance. ## DIWYDU: Automating Include Cleanup - Google’s Include What You Use (IWYU) tool was considered but proved difficult to apply retroactively to Figma’s large codebase. - Figma created a less strict alternative called **Don’t Include What You Don’t Use (DIWYDU)**. - DIWYDU: - Uses Python bindings for `libclang`. - Parses source and header files into Clang Abstract Syntax Trees. - Identifies types, functions, and variables directly used by each file. - Flags headers that are included but provide no directly used symbols. - The tool runs on feature branches to prevent unnecessary includes from accumulating. ## DIWYDU’s Limitations - It analyzes Figma-owned files but excludes Standard Template Library headers. - STL headers may define symbols through private internal includes, making direct dependency analysis difficult. - Python’s `libclang` bindings expose less of Clang’s AST than the compiler’s native C++ APIs, sometimes producing `UNEXPOSED_EXPR` nodes. - A future C++ implementation could provide more accurate AST access. - DIWYDU cannot detect cases where an included header is genuinely required but excessively large. - Such regressions may need forward declarations or header decomposition instead. ## Measuring Dependency Growth with `includes.py` - Figma built `includes.py` to measure the transitive bytes associated with each source file. - The tool is written entirely in Python and typically runs in a few seconds without invoking Clang. - It: - Crawls first-party source, header, and generated files. - Counts file sizes. - Builds a dependency graph. - Estimates the total bytes passed to the compiler for each source file. - Standard library includes are treated as zero bytes because Figma mainly accesses them through internal wrapper directories. - CI uses the measurements to compare pull requests and warn authors when changes significantly increase compiled bytes. Figma’s approach demonstrates that controlling header dependencies can deliver larger and more durable gains than simply adding hardware or cache capacity. Teams working on large C++ codebases should automate unused-include checks, measure transitive dependency size in CI, and use forward declarations or smaller headers when necessary.

Read original(opens in new tab)

Static AnalysisCI/CDPythonC+++4