Code Review

32 posts

githubAug 11, 20262 min readCurated summary

From coder to orchestrator: How agents shift the role of a developer

AI agents can generate impressive one-prompt demos, but reliable software delivery requires more than isolated outputs. Developers increasingly need to design workflows that define how code is proposed, tested, reviewed, and shipped. The article argues that this shifts developers from primarily writing code to orchestrating agents within controlled, repeatable systems. ## From One-Off Prompts to Reliable Workflows - A single prompt can quickly produce a demo, such as a simple game. - Production development requires repeatable delivery with: - Appropriate context - Validation and testing - Security controls - Review processes - Clear permissions and handoffs - GitHub Copilot is presented as a control plane for connecting these parts. ## An Agentic Development Flow - Familiar repository events can trigger agent work, including: - Adding a label to an issue - Running a scheduled workflow - Starting a GitHub Actions process - The agent’s changes are captured in a pull request. - Deterministic checks then validate the work through: - Linting - Tests - Security scans - Build verification - CODEOWNERS, required reviews, and branch protection rules control what can be merged. - Agents handle ambiguous, context-heavy tasks, while predictable automation provides the safety boundary. - Developers decide: - What agents can access - How tasks are scoped - Where workflows hand off - When human judgment is required ## GitHub’s Implementation Options - Copilot cloud agent workflows support event-driven automations. - Copilot CLI can run AI-powered steps inside GitHub Actions. - Model Context Protocol (MCP) can extend agents with additional tools and external context. - These options represent different stages of building an agent-enabled development workflow. ## Starting Small - Teams should begin with one bounded, low-risk workflow. - Suitable examples include: - Issue triage - Synchronizing documentation and tests - Routine maintenance updates - The recommended approach is to integrate Copilot into existing development infrastructure rather than redesigning everything at once. Developers should treat AI agents as components within an engineered delivery system, not as replacements for that system. Start with a limited workflow, surround agent output with automated checks and review controls, and gradually expand as the process proves reliable.

Code Review AI Agents CI/CD Model Context Protocol+4

lineAug 7, 20264 min readCurated summary

What Is the Next Step in Personal AI Use? Conditions for Introducing an AIDD Organization Explored Through an AIDD Workshop at LY Corporation

LY Corporation argues that AI-driven development (AIDD) must evolve beyond individual experimentation into a repeatable organizational practice. AIDD integrates AI across requirements, design, implementation, and review, with AI producing drafts while people provide context, make decisions, and maintain accountability. Its workshop showed that successful adoption depends less on distributing tools than on preparing shared context, workflows, responsibilities, and decision-making structures. ## Defining AIDD - AIDD uses AI as a collaborator throughout the development lifecycle, from requirements clarification through code review. - It is neither fully delegating development to AI nor using AI as an isolated productivity assistant. - The intended workflow is: - AI creates an initial draft. - People provide intent, constraints, and judgment. - Results are reviewed and carried into subsequent development stages. - The central challenge is designing how people and AI work together across the entire process. ## Why LY Corporation Held the Workshop - Individual use of AI coding agents has become common for: - Code completion - Research - Testing - Documentation - Organizational adoption often stalls because: - Individual usage is not connected to team workflows. - Review standards for AI output are unclear. - Teams are unsure how to apply AI to existing products. - Successful experiments remain personal know-how. - “Convenience” does not translate into investment or adoption decisions. - The workshop aimed to move teams from personal AI usage toward organization-wide “AI Ready” conditions. - It involved 21 teams and 112 participants, including LINE Plus, who brought real work topics for evaluation. ## Why Participation Was Team-Based - AI creates value through workflow design, not just prompt-writing skill. - Teams must decide: - What information AI receives - Where human review occurs - Which output becomes the official deliverable - How feedback enters the existing process - Engineers alone cannot resolve these questions. Product, planning, design, leadership, and decision-makers contribute essential perspectives. - Team participation exposed hidden disagreements about consensus, ownership, review responsibilities, and decision boundaries. ## Workshop Structure - The two-day program combined learning with practical validation using real team projects. - Day one focused on: - Defining problems - Organizing requirements and context - Clarifying assumptions and priorities - Day two focused on autonomous experimentation and producing workflows applicable to actual work. - Orchestration Guild members, Developer Relations, and Technical Directors provided mentoring and helped make the learning reproducible across the company. - Informal conversations during breaks and meals also helped reveal issues and next steps that formal meetings often miss. ## Four Major Lessons ### The Greatest Value Often Comes Before Implementation - Teams initially focused on how quickly AI could write code. - In practice, the more important benefits came earlier in the process: - Breaking vague requirements into concrete issues - Defining requirements in clear language - Aligning team understanding - Identifying which decisions must come first - Turning decisions into manageable work units - AI can accelerate progress, but people must establish the problem definition and make critical judgments. ### Context, Not Tools, Is the Main Bottleneck - AI output quality depends heavily on the quality of its context. - Important context includes: - Specifications - Terminology - Constraints - Design intent - Relationships to existing code - Operational rules - Without this information, AI may generate plausible but impractical results, increasing review effort. - Organizing context must therefore be treated as core infrastructure for AI adoption, not optional preparation. ### Team Participation Reveals Organizational Issues - Individual experiments rarely expose the full set of coordination problems. - Working on a shared topic helps teams determine: - Where AI should be used - Who reviews its output - Which artifacts are authoritative - How AI-assisted work fits into existing processes - Collaboration across business, planning, design, engineering, and leadership makes implicit knowledge and conflicting assumptions visible. ### Decision-Maker Involvement Improves Follow-Through - Teams with leaders or decision-makers were more likely to turn workshop outcomes into concrete actions. - Organizational adoption requires decisions about: - Which areas to start with - Where to invest time - What to standardize - How deeply AI should be embedded into operations - Leadership participation prevents the workshop from ending as an interesting experiment and helps connect it to implementation. ## Conditions for Successful Adoption - Start with manageable topics, such as: - Requirements or issue clarification - Work requiring stakeholder alignment - Projects with accessible existing information - Small efforts where one complete cycle can be tested - Create lightweight entry points, such as applying AI to one feature, one requirements document, or one review checklist. - Make context preparation an official responsibility: - Document specifications, terminology, constraints, design intent, and decision rationale. - Allocate team and organizational time for this work rather than relying on individual goodwill. - Treat context organization as a long-term engineering asset that improves development even beyond AI use. The practical recommendation is to adopt AIDD incrementally through real team projects, while simultaneously improving shared context, review processes, role definitions, and leadership involvement. The goal is not merely to use more powerful tools, but to redesign the development system so AI-assisted work can be repeated and sustained across the organization.

Code Review Software Development Software Design Context Engineering+4

cloudflareAug 5, 20263 min readCurated summary

How we’re rethinking work at Cloudflare with Cloudflare OS

Cloudflare built Cloudflare OS to let employees use AI agents safely after a sudden increase in demand for production access and automation capabilities. The company’s approach combines AI enablement with strict controls around data access, human accountability, organizational context, and engineering quality. Its experience suggests that successful AI adoption requires meeting both technical and non-technical users where they work. ## Why Cloudflare Built Cloudflare OS - Employees rapidly began using improved AI models and agent-building tools to create internal applications. - One sales employee requested production access to roughly a dozen systems and administrative deployment permissions for an AI-built “SuperApp.” - Cloudflare needed to enable experimentation without exposing internal systems, company data, or customer data. - The resulting platform combines existing products such as Workers and Access with custom services developed for internal AI workflows. ## Principles for AI Adoption - **Start with jobs to be done:** Teams should identify customer-related pain points, bottlenecks, or missed opportunities before selecting an AI tool. - **Give everyone access to AI capabilities:** AI interfaces should not be limited to developers using terminals, code editors, and repositories. - **Keep humans accountable:** Employees remain responsible for defining quality, testing outputs, and owning the workflows and agents they deploy. - **Prioritize organizational context:** Cloudflare-specific knowledge and canonical internal guidance matter more than simply choosing the most powerful model. - **Never expand permissions through AI:** AI tools and agents must inherit users’ existing access restrictions and receive only the permissions required for their tasks. Shared agents must respect each recipient’s permissions rather than the deployer’s. ## Engineering Guardrails with the Cloudflare Engineering Codex - Cloudflare created the Engineering Codex as an authoritative, opinionated guide to engineering practices. - Unlike policies, which define what engineers cannot do, the Codex describes what they should do. - Domain owners are responsible for defining quality standards across the codebase. - AI agents use the Codex throughout the software development lifecycle: - Planning work - Reviewing merge requests - Evaluating technical designs before implementation - Reviewing incident reports - Over four months, these agents identified nearly 250,000 potential issues, blocked 16,000 merges, and caught architectural problems in almost 600 designs. - Cloudflare is now focusing on helping engineers create evaluation loops for assessing the work produced by their agents. ## Rethinking AI Tools for Non-Engineers - Cloudflare initially gave non-engineering employees developer-oriented tools with more approachable interfaces. - This approach worked poorly for knowledge workers who create one-off deliverables and interact with many systems of record. - Code-focused harnesses encouraged excessive “vibe-coded” applications, often without a clear problem to solve. - Cloudflare then began working backward from users’ actual needs and introduced the idea of a “magic AI email bot” to which employees could delegate unwanted work. The supplied excerpt ends before describing how that system worked. Cloudflare’s experience recommends pairing broad AI access with strong identity, permission, context, and accountability systems. Organizations should design tools around real jobs to be done—not simply distribute coding agents—and provide interfaces suited to both engineers and non-engineers.

Code Review AI Agents Cloudflare Workers Zero Trust+2

githubJul 27, 20262 min readCurated summary

GitHub Copilot app for Beginners: Getting started

The GitHub Copilot app is designed as a development workspace rather than a single AI chat window. It connects agent sessions to projects, supports parallel tasks, provides an interactive browser canvas for UI work, and helps manage pull requests through Agent Merge. Together, these features aim to support the full workflow from exploration to shipping. ## Project-Based Agent Sessions - Each session is connected to a specific project and its repository context. - Projects can be selected from GitHub or added from a local machine. - Copilot can inspect the codebase, identify relevant files, implement changes, and run tests. - This reduces the setup required before beginning a development task. ## Managing Multiple Work Threads - Users can create separate sessions for different tasks without interrupting ongoing work. - **Quick Chat** provides a lightweight way to: - Ask questions about Copilot or the codebase - Explore implementation options - Investigate unfamiliar parts of an application - Gather context before making changes - Returning to an existing session preserves its history and allows work to continue from where it stopped. ## Interactive UI Work with Canvas - The app includes a browser canvas for previewing applications alongside the AI conversation. - Canvas can be created with the `/create-canvas` slash command. - **Enable Canvas Dev Mode** and **Pick & Polish** allow users to select page elements directly and use them as context for refinement requests. - This supports an iterative workflow in which developers can inspect the visual result, identify problems, and ask Copilot to adjust specific UI elements. ## Pull Request Assistance with Agent Merge - **Agent Merge** extends Copilot’s role beyond implementation into code review and delivery. - It can be enabled from a pull request’s options in the Copilot app. - Developers choose which actions it may perform, including: - Addressing review feedback - Helping resolve CI failures - Handling merge conflicts - Agent Merge monitors the pull request while checks and reviews are in progress, preparing it for merge once requirements are satisfied. The Copilot app is intended to centralize development activities in one workspace: start with a project, separate work into focused sessions, visually refine applications through canvas, and use Agent Merge to help complete the pull request process. Developers can learn the workflow by applying it to an existing backlog task.

Code Review AI Agents CI/CD Github+3

gitlabJul 27, 20263 min readCurated summary

Claude Opus 5 on GitLab: Reasoning built for the hard tasks

Claude Opus 5 is now available through GitLab Duo Agent Platform for complex, high-stakes engineering tasks. GitLab reports a 93.3% benchmark resolution rate, significantly above Opus 4.8’s 73.0%, while maintaining comparable or better speed. The model is intended for difficult debugging, large refactors, coordinated multi-agent work, and code reviews where correctness matters more than minimizing initial cost. ## Reliable Reasoning for Complex Work - Opus 5 is designed to maintain consistent reasoning across long-running tasks and multiple steps. - It aims to reduce partial patches, failed runs, diagnostic work, and repeated prompting. - In GitLab’s internal evaluation: - Opus 5 completed 100% of the tasks it attempted. - Its verified resolution rate was 93.3%, compared with 73.0% for Opus 4.8. - In one five-file SSO authentication task, Opus 5 implemented the feature, added exported types and configuration fields, committed the changes, and opened a merge request. - The model is also presented as effective for code review, identifying genuine bugs while producing relatively few false positives. ## Coordinating Multiple Agents - Opus 5 can support workflows where several agents work in parallel. - Subagent coordination helps prevent agents from interfering with one another’s changes. - Writer-verifier patterns allow one agent to check another’s output before it is accepted. - GitLab Credits usage caps can limit spending when running multiple parallel agents. ## Speed Alongside Depth - On GitLab’s most difficult benchmark tasks, Opus 5 was faster at the 95th percentile: - 768 seconds for Opus 5 versus 784.98 seconds for Opus 4.8. - 768 seconds versus 982.57 seconds for Sonnet 4.6. - GitLab argues that this combination of speed and reliability produces more predictable completion times for long tasks. ## Choosing the Appropriate Model - Sonnet-class models remain suited to routine development because they are fast, affordable, and dependable. - Opus 5 is recommended for: - Difficult debugging - Large refactors - Complex multi-file features - Decisions where rework would be costly - Model selection is configurable within GitLab, while all models use the same context management, policy controls, and audit trail. ## Availability - Claude Opus 5 is available on GitLab Duo Agent Platform and uses GitLab Credits. - New users can start with a free trial. - GitLab Premium and Ultimate subscribers can enable Duo Agent Platform and use included credits. Opus 5 is best viewed as a specialized model for high-complexity work rather than a universal replacement for faster Sonnet-class models. Teams should match the model to task difficulty and use spending limits when scaling autonomous, parallel-agent workflows.

Code Review AI Agents Refactoring Gitlab Duo Agent Platform+4

figmaJul 23, 20263 min readCurated summary

How Figma Stays Ahead of Vulnerabilities With Agents | Figma Blog

Figma uses AI agents to prevent, detect, and fix vulnerabilities during code generation, pull-request review, and historical code audits. Its central strategy is a shared security policy combined with continuous measurement of precision and recall. The company began with PR review because developer feedback and replaying known bugs created the fastest improvement loop. ## A Shared Policy Across the Development Lifecycle - The same policy guides agents during: - Code generation - Pull-request review - Full-repository auditing - The policy records: - Trust boundaries - Accepted risks - Security precedents - Improving PR review first helped Figma refine the policy before applying it to other stages. ## Measuring Precision and Recall - **Precision** measures how many reported findings are genuine vulnerabilities; low precision creates false positives. - **Recall** measures how many real vulnerabilities the system detects; low recall creates false negatives. - Figma measures: - Precision through author thumbs-up or thumbs-down feedback on findings. - Recall by replaying the reviewer against commits containing known bugs. - These separate signals allow the team to improve both dimensions rather than optimizing for only one. ## AI-Assisted Pull-Request Review - Every pull request receives an automated review. - Findings are posted directly to the PR, allowing developers to respond and fix issues in context. - Figma currently runs: - Claude Code with Opus 4.8 at extra-high effort - Codex with GPT-5.6 Sol at high effort - The models identify different classes of bugs, so Figma reports a finding if either model detects one. - Reviews cost approximately $0.50 per pull request at the median. The agents have identified both sophisticated and conventional vulnerabilities, including: - A multi-step desktop-client exploit in which an injected sandbox object exposed the host realm’s `Function` constructor and enabled possible code execution. - An insecure direct object reference where an authenticated user could retrieve another organization’s invoice by supplying its ID. ## Building Trust Before Exposing Findings - Figma launched Anthropic’s Claude Code Security Reviewer in August 2025 in shadow mode. - Initial results were strong at reproducing known vulnerability root causes, but only 4 of 27 findings—about 15%—were valid. - The team prioritized precision first because developers quickly lose trust in tools that generate excessive false positives. - Figma set a practical target of at least 70% precision. - Developer-facing comments were withheld until precision exceeded that level over a two-week period without severe false positives. - Security engineers replayed the reviewer across eight weeks of historical pull requests and manually labeled incorrect findings. - Those examples were used to create and refine the shared security policy. ## Continuous Improvement Through Precedents - A **precedent** documents why a finding is valid or invalid in a particular context. - Human feedback and historical vulnerability replays feed back into the policy. - This process lets Figma improve the agent automatically rather than relying only on model changes or one-time prompt tuning. Figma’s approach treats agentic security as an engineering and measurement problem, not simply a matter of asking an AI to scan code. Organizations adopting similar systems should establish feedback loops, measure precision and recall independently, and build developer trust before making automated findings part of everyday development.

Code Review AI Agents Claude Code Monorepo+3

gitlabJul 16, 20263 min readCurated summary

Forrester Consulting: GitLab Duo Agent Platform delivers 400% ROI

GitLab-commissioned Forrester research found that organizations using GitLab Duo Agent Platform could achieve a 400% three-year ROI, $7.5 million in net present value, and payback in under six months. The study argues that agentic coding creates the greatest business value when integrated across the software lifecycle—not merely used to generate code. Benefits included faster onboarding, shorter migrations, quicker security remediation, and more developer time for feature work. ## Study Scope and Financial Model - Forrester interviewed four organizations across financial services, software, entertainment, and insurance. - Their experiences were modeled as a composite global company with: - $3 billion in annual revenue - 3,000 employees - GitLab Duo Agent Platform adoption growing from 150 to 250 users - Three-year risk-adjusted costs totaled approximately $1.9 million: - $1.3 million in consumption credits - $589,000 for implementation, training, support, and internal labor - Quantified benefits reached $9.4 million, producing: - 400% ROI - $7.5 million net present value - Payback in less than six months ## Problems Before Adoption - Manual processes and ad-hoc knowledge sharing slowed development. - New developers depended heavily on senior engineers for context and troubleshooting. - Security fixes waited in queues for specialists with the necessary expertise. - Code review was often a larger bottleneck than writing code. - Senior engineers were repeatedly interrupted to unblock other team members. ## Quantified Benefits - **80% faster developer onboarding** - Agentic chat in IDEs and repositories helped new hires understand unfamiliar codebases independently. - Estimated savings: $582,000. - **75% shorter migration timeline** - A migration from on-premises GitLab to GitLab SaaS finished in two months instead of eight. - Agents helped diagnose pipeline failures and resolve issues during the migration. - Estimated labor savings: $157,000. - **40% more time for security and QA engineers** - Contextual explanations and suggested fixes reduced remediation effort and reliance on senior staff. - Estimated three-year savings: $1.3 million. - **20% more developer capacity for feature work** - Agents supported code review, testing, and troubleshooting. - Estimated combined benefit: $7.4 million. - Additional unquantified benefits included reduced need for overlapping AI tools, improved developer satisfaction, and stronger knowledge sharing. ## Broader Impact on Software Delivery The study found that organizations shipped features in days rather than weeks, resolved vulnerabilities more quickly, onboarded staff faster, and compressed major infrastructure work. Its central conclusion is that AI productivity gains compound when agentic coding is connected to infrastructure supporting the entire software delivery lifecycle. The findings are based on interviewed organizations and a composite financial model, so actual results will vary. Companies evaluating agentic development should use the study as a framework for estimating benefits across productivity, security, onboarding, and operational efficiency—not as a guaranteed ROI.

Code Review AI Agents CI/CD DevSecOps+3

githubJul 10, 20263 min readCurated summary

Better tools made Copilot code review worse. Here’s how we actually improved it.

Copilot code review became more expensive and less effective after GitHub replaced its specialized exploration tools with shared `grep`, `glob`, and `view` tools. The tools themselves worked correctly, but their general-purpose instructions encouraged broad repository browsing rather than focused pull request investigation. After rewriting the instructions around diff-first review workflows, GitHub achieved roughly 20% lower average review cost without reducing review quality. ## Why the Tool Migration Regressed - Copilot code review previously used specialized tools for: - Listing directories - Searching files and directories - Reading code - These tools often returned matching lines along with surrounding context, which suited earlier models that made fewer tool calls and needed more context per request. - GitHub migrated to the shared Unix-inspired tools used by Copilot CLI and other products: - `glob` replaced `list_dir` - `grep` replaced `search_file` and `search_dir` - `view` replaced `read_code` - The migration aimed to reduce duplicated implementations and let improvements benefit multiple Copilot products. - Offline benchmarks showed higher review costs and fewer useful comments after the migration. ## Repository Browsing Instead of Pull Request Review - Execution traces showed the agent: - Searching broadly - Guessing file paths - Reading large sections of code - Finding more things to search - Carrying unnecessary context into later reasoning - This workflow is reasonable for a coding assistant asked to understand an unfamiliar repository. - It is inefficient for code review, where the agent should begin with the pull request diff and investigate a specific potential problem. - Excessive tool output increases token usage because returned file contents remain in the agent’s context window. - Broad exploration can also make the review less focused by mixing relevant evidence with unrelated code. ## The Difference Between Coding and Reviewing - A coding assistant may need to map a large area of a repository before editing code safely. - A reviewer typically asks targeted questions based on the diff, such as: - Where is the changed function called? - Is a modified configuration key used elsewhere? - Does a similar test or helper already exist? - What is the smallest code range needed to understand the behavior? - Copilot code review has a narrower objective: - Start from the pull request diff - Determine whether the change introduced a real issue - Gather only the evidence needed to confirm or dismiss that issue - The shared tools were designed for broader interactive workflows, so their instructions unintentionally encouraged the wrong behavior in the review agent. ## Instructions Were the Real Fix - GitHub concluded that changing tools was not enough; the agent’s workflow instructions also had to change. - The revised instructions emphasized: - Diff-first investigation - Targeted searches - Minimal surrounding context - Narrow evidence gathering - Avoiding unnecessary repository-wide exploration - With these workflow changes, the shared tools became more effective for review rather than merely reproducing their coding-assistant behavior. - The result was approximately 20% lower average review cost while preserving review quality. The practical lesson is that tool quality cannot be evaluated separately from the instructions and workflow guiding an agent. Shared tools can work well across products, but each use case needs instructions that match its task—in this case, focused, evidence-driven pull request review rather than broad repository exploration.

Code Review AI Agents Github Copilot Pull Requests+2

lineJun 25, 20264 min readCurated summary

What If AI Agents Debated Each Other? Redesigning the Development Process Through Multi-Agent Collaboration

AI coding’s main bottleneck is no longer code generation but the human coordination surrounding it: clarifying intent, validating assumptions, testing implementations, and preparing trustworthy pull requests. LY Corporation proposes an AI-native pipeline in which specialized “proposer” and “challenger” agents debate across three stages—specification, build, and delivery—while an orchestrator decides whether to revise, escalate, or proceed. The goal is for AI to substantiate its own work before human engineers review and approve it. ## Human Coordination as the Bottleneck - Traditional AI-assisted development speeds up individual tasks but leaves handoffs between requirements, implementation, verification, and review to humans. - Engineers still need to: - Write or refine specifications - Review AI-generated drafts - Transfer failed tests and feedback between steps - Inspect diffs - Prepare PR descriptions - Decide whether the result is trustworthy - The proposed solution is not to remove human judgment, but to automate repetitive coordination while preserving human ownership and final approval. ## Proposer–Challenger Collaboration - AI responsibilities are divided between two opposing groups: - **Proposers** develop specifications, implementations, and delivery materials. - **Challengers** validate them from specialized perspectives. - The separation prevents one general-purpose assistant from combining design, implementation, testing, and review into a single unchallenged response. - Specialized roles may include: - `requirements-synthesizer` - `security-analyst` - `test-coverage-reviewer` - `technical-writer` - `evidence-verifier` - An **orchestrator** mediates disagreements, redirects discussions, resolves deadlocks, and determines whether to revise, escalate, or advance. ## The Spec–Build–Deliver Pipeline ### Specification - The specification acts as a contract for all later stages. - It records: - Goals and constraints - Interpreted requirements - Explicit assumptions - Open questions - Proposed approach - Definition of done - Agents use evidence from the workspace and external sources such as Jira, Confluence, design documents, APIs, tests, dependencies, and existing conventions. - Ambiguous but low-risk and reversible issues can be documented as assumptions. - Unsafe, destructive, externally constrained, or hard-to-reverse uncertainties are escalated instead of guessed. ### Build - The approved specification is converted into a test-first verification plan before production code is changed. - The proposer identifies expected behavior, edge cases, required tests, and execution commands. - Challengers can dispute the verification design before or during implementation. - Proposers must support rejected objections with concrete evidence such as: - Execution paths - Compiler or linter output - Failing tests - Other workspace evidence - This prevents a simple green CI result from hiding missing or inadequate validation. ### Delivery - The final output is a review-ready PR package rather than merely a diff summary. - It explains: - What changed - Where reviewers should look first - Which checks passed - Remaining risks - Which challenges were already investigated - At this stage, the orchestrator acts more like a jury, judging whether sufficient evidence exists for release. ## Structured Debate Protocol - Each agent receives stage-specific context and returns structured JSON rather than a free-form essay. - Agents do not share one live context window. Shared state consists of: - Workspace files - Generated artifacts - The orchestrator’s accumulated transcript - Each round includes a proposer response, challenger response, and orchestrator decision. - The protocol distinguishes manageable uncertainty from blocking risk. - Consistent schemas make agent outputs easy to parse, compare, and feed into subsequent rounds. - For example, a challenger can identify an unclear scope boundary, explain why it matters, assign severity and confidence, and indicate whether user input is required. ## Overall Impact - Issues move through a continuous chain: debated specification, branch, tested implementation, and review-ready PR. - Humans intervene mainly to define intent, approve the final result, or resolve explicitly escalated decisions. - The central leverage comes not from generating code faster, but from requiring AI to explore, challenge, verify, and package its work before asking engineers to pay attention. The practical recommendation is to redesign AI development around explicit artifacts, specialized adversarial roles, evidence-based decisions, and automated handoffs. Human engineers should remain the final decision-makers, while AI handles the intermediate coordination and proof-building work.

Code Review AI Agents CI/CD Json+3

awsJun 17, 20263 min readCurated summary

AWS Security Agent adds threat modeling, Kiro power and Claude Code plugin, and more | Amazon Web Services

AWS Security Agent, now part of AWS Continuum, is expanding into a unified security platform spanning design, development, and deployment. New capabilities include threat modeling, deeper pull-request and repository reviews, compliance validation, and integrations with Kiro, Claude Code, and other AI IDEs through MCP. AWS’s central promise is to find vulnerabilities, verify exploitability, and provide actionable fixes without interrupting developers’ workflows. ## Enhanced Code Reviews - Supports GitHub, GitLab, and Bitbucket, including SaaS and self-hosted installations. - Integrates with Confluence so reviews can use existing technical and security documentation as context. - Performs reasoning-based analysis of pull requests and entire repositories, identifying vulnerabilities beyond pattern matching. - Checks code against organizational security requirements and common security risks. - Validates findings in simulated environments to demonstrate proof of exploitability. - Provides remediation guidance and fix commits directly in the repository workflow. - Supports pull-request scanning, remediation, security requirement packs, and simulated validation. ## Compliance and Design Reviews - Managed compliance packs include: - AWS WAF - NIST CSF - PCI DSS - AWS best practices - Organizations can import their own security requirements from internal documents or Confluence. - Findings are mapped to the organization’s compliance posture, helping teams maintain audit readiness throughout development. ## Threat Modeling - Generates threat models from design documents or source-code repositories. - Builds an application context model covering: - Architecture - Data flows - Trust boundaries - Application components - Identifies likely threat actors, attack vectors, and weaknesses. - Uses the STRIDE framework to classify threats. - Prioritizes risks and recommends mitigations. - Threat models can be generated from the Security Agent console or through supported development tools. ## Kiro, Claude Code, and MCP Integrations - A Kiro power is available, with a Claude Code plugin coming soon. - An open MCP integration enables use from Kiro, Claude Code, or other AI-powered IDEs. - Developers can trigger scans, threat models, and remediation directly from their IDE or CLI. - Kiro commands include: - `Set up AWS Security Agent` - `Run a full security scan on this repo` - `help me remediate my findings` - `Build a threat model for this application` - Findings can be downloaded into the local workspace, prioritized, and used to start bug-fix specification sessions. - Generated threat models are saved at `.security-agent/threat_model.md`. - A Kiro agent hook can automatically evaluate whether a code-review scan should run after an agent turn. ## Lifecycle Coverage and Availability - Design-time security: design reviews and threat modeling, both in preview. - Development-time security: repository and pull-request code reviews, in preview. - Deployment-time security: on-demand penetration testing, generally available. - The features are available in supported AWS commercial Regions, with regional availability and pricing listed in AWS documentation. AWS Security Agent is positioned as a continuous security layer across the software lifecycle. Teams interested in automated, context-aware security analysis can try it through the Security Agent console, while developers can use the IDE and MCP integrations to keep scanning and remediation within their existing workflows.

Code Review Model Context Protocol Claude Code Threat Modeling+3

dropboxJun 12, 20263 min readCurated summary

How Dropbox uses MCP and Dash to close the design-to-code security gap

Dropbox found a significant gap between security design reviews and implementation. Only 12% of implementing pull requests linked back to their original threat models, and the median delay between review and code submission was about five weeks. To close this gap, Dropbox built a system using Dash, Model Context Protocol (MCP), and foundational models to automatically retrieve relevant security requirements and compare them with code during review. ## The Design-to-Code Gap - Threat models document risks, attack scenarios, and agreed-upon mitigations during security review. - These documents often remain in wikis or documentation systems, while implementation happens later through pull requests. - At Dropbox: - Only 12% of implementing PRs linked to their original design review. - Among 79 verified pairs, 54% of PRs were opened more than a month after the review. - The median delay was approximately five weeks, with some delays exceeding 11 months. - Only 29% of PRs were opened within two weeks of the security review. - About 15% of design reviews were filed retroactively, suggesting that some security-sensitive work was not identified early enough. ## Why Existing Tools Fall Short - Static analysis can detect whether certain security patterns or controls exist. - It generally cannot determine whether those controls satisfy the specific requirements agreed upon during design review. - Manual linking between PRs and threat models depends on engineers remembering additional workflow steps. - Reminder bots may improve compliance temporarily, but adherence tends to decline. - The core problem is not a lack of security documentation; it is the difficulty of making that existing context available during implementation. ## Dash and MCP as a Context Bridge - Dash already indexes Dropbox content and connected applications, including threat models and engineering documentation. - Dropbox used Dash’s MCP server to let an AI security agent search and read this content. - MCP provides a common interface for bringing multiple context sources into one agent session. - When a PR is opened, the agent retrieves relevant threat models and supporting documents. - A foundational model then compares the documented requirements with the proposed code. - For example, it can identify whether code implementing an endpoint enforces authentication required by the corresponding threat model. - Unlike traditional static analysis, this approach evaluates code against documented security intent, not just known code patterns. ## Integrating Security into Code Review - Dropbox surfaced the system directly within the existing code-review workflow rather than creating a separate security process. - This allows developers and reviewers to receive relevant security context where implementation decisions are already evaluated. - The same mechanism can potentially identify missing reviews when code appears security-sensitive but has no corresponding threat model. Dropbox’s approach shows how retrieval and AI reasoning can reconnect design decisions with implementation. Organizations can apply the same pattern beyond security—for example, to compliance requirements, privacy reviews, accessibility standards, or other design-to-code checks.

Code Review Large Language Models Model Context Protocol Static Analysis+2

gitlabJun 9, 20262 min readCurated summary

Mythos-class Claude Fable 5 arrives on GitLab Duo Agent Platform

Claude Fable 5, Anthropic’s Mythos-class model, is now available through GitLab Duo Agent Platform across all tiers and deployment models. The post presents it as a major step beyond incremental model improvements, emphasizing stronger first-attempt accuracy, sustained autonomous workflows, and better code review and incident analysis. Its main conclusion is that teams should apply it to complex, previously difficult engineering problems rather than routine tasks. ## More Accurate First-Pass Results - Produces more correct implementations for complex, well-defined problems with fewer iterations. - Reduces back-and-forth in Duo Agentic Chat. - Performs especially well on: - Multi-file refactoring - Incident investigation - Infrastructure-as-code definitions - Interprets technical images, web applications, and detailed screenshots more accurately while often using fewer output tokens. ## Longer and More Reliable Agent Workflows - Sustains goal-directed work across extended, multi-day runs and millions of tokens. - Maintains instructions and task focus without frequent manual checkpoints or re-prompting. - Uses verification loops to identify and correct its own mistakes. - Handles parallel sub-agents more reliably across multiple repositories or services. - Reduces the amount of human oversight required per agent run, allowing teams to review results asynchronously. ## Improved Bug Detection and Incident Response - Provides higher bug-finding recall than previous models. - Improves outage triage, root-cause analysis, and repository-history investigation. - Finds deeper code-path issues and edge cases during merge request reviews. - Produces more actionable review comments instead of generic feedback. - Helps reduce production defects and improve mean time to resolution. ## Recommended Use Cases - The post recommends testing the model on difficult, unsolved engineering problems rather than routine work. - Suggested applications include: - Complex multi-file refactors - Production incident investigations - Implementations developers would normally write manually - Teams should allow agents to scope work, ask clarifying questions, and execute multi-step solutions. Claude Fable 5 became available on GitLab Duo Agent Platform on June 9, 2026. Teams can access it through a free trial, the GitLab free tier, or included GitLab Credits for Premium and Ultimate subscribers.

Code Review AI Agents CI/CD DevSecOps+4

dropboxMay 28, 20263 min readCurated summary

Beyond code generation: rethinking engineering productivity in the age of AI agents

AI coding agents have increased Dropbox’s code production, but they have also exposed bottlenecks in review, testing, release coordination, and operations. The central argument is that engineering productivity must be measured by end-to-end product velocity and customer impact—not code or pull-request volume alone. Dropbox is responding with agent platforms, stronger workflow infrastructure, broader quality metrics, and training that helps engineers adapt to new responsibilities. ## From Copilots to Agents - Copilots assist with explanations, snippets, and questions within existing workflows. - Agents can take scoped tasks, inspect repositories, edit files, run tests, fix failures, and return changes for human review. - Engineers remain responsible for intent, architecture, quality, and release decisions. - Increased parallel work and code output create new pressure on: - Code review systems - CI and testing infrastructure - Validation workflows - Release processes - Production operations - More code and pull requests do not necessarily create more customer value. ## Nova as Dropbox’s Agent Platform - Nova runs AI coding agents in controlled environments with relevant codebase context and internal engineering practices. - Its value comes from the surrounding platform—safe execution, workflow integration, guardrails, and human review—not only from the underlying model. - Nova generates approximately one in twelve Dropbox pull requests. - It supports both feature work and maintenance tasks, including: - Migrations - Flaky-test remediation - Bug investigation - Dependency updates - Other high-effort engineering work - The intended workflow is structured: define the task, let the agent work within constraints, validate the result, and require human approval before production. ## Measuring Product Velocity and Impact - Pull-request throughput was useful when implementation speed was the main constraint, but it is insufficient as AI increases output. - Dropbox evaluates whether the broader engineering system can absorb additional work efficiently. - Its four-stage measurement model tracks: - **Fuel:** Whether engineers use AI tools - **Adoption:** How teams change their workflows - **Output:** Whether AI contributes to production work - **Impact:** Whether products reach customers faster and create greater value - Quality and trust metrics include review turnaround time, first-run test pass rate, defect ratio, and rework rate. - Productivity improvements must not come at the expense of reliability or customer trust. ## Evolving Engineering Workflows - Engineers increasingly focus on defining intent, framing problems, reviewing changes, and making architectural and quality decisions. - Adoption requires more than tools; Dropbox uses hands-on learning, hackathons, bootcamps, workflow examples, and peer-led training. - Teams should adopt agents at different speeds based on risk, context, and readiness. - The goal is not to automate every workflow, but to make agentic development safe, useful, measurable, and repeatable. ## Broader Lessons - AI does not remove bottlenecks; it shifts them downstream. - Organizations must invest in validation, orchestration, governance, workflow integration, and measurement alongside code generation. - Competitive advantage will come less from access to common foundation models and more from the systems built around them: internal context, safeguards, quality controls, and integrated engineering workflows. Dropbox’s experience suggests that companies should treat AI agents as a change to the entire software delivery system, not merely as faster coding tools. The strongest results will come from improving the full path from idea to validated production impact.

Code Review AI Agents CI/CD Automation+3

gitlabMay 28, 20263 min readCurated summary

Agentic coding is only as good as its context

Coding agents are only as effective as the context they can access. Repository-only agents may produce compiling fixes, but they often miss issue requirements, CI rules, security policies, and review standards. Integrating agents with GitLab’s issues, merge requests, pipelines, and security controls produces better-aligned code, fewer review cycles, and safer releases while preserving human approval. ## From Repository Context to Full Lifecycle Context - GitLab tutorials demonstrate progressively richer workflows using Claude Code and Codex. - **Repository only:** The agent reads local files, interprets the prompt, and runs the build, but lacks acceptance criteria, organizational requirements, and review expectations. - **Repository plus issue:** Through GitLab’s MCP server, the agent can read issue descriptions, labels, milestones, and implementation notes. - It can correctly link changes to issues, such as adding `Closes #32`. - Tools such as `get_issue` and `create_merge_request` connect implementation work to planned work. - **Working inside the merge request:** GitLab’s Code Review Flow provides feedback, which the agent can address by adding tests, improving documentation, and fixing validation gaps. - New commits automatically trigger CI/CD. - Human reviewers can inspect the result in the same merge request. - The tutorials report fewer review rounds and shorter merge times. ## Why Platform Visibility Matters - Platform teams determine: - Which agents and tools are permitted - How agent output is verified - Where human approval is required - Important context lives in the DevSecOps platform: - Issues define requirements. - CI/CD configuration defines quality standards. - Review instructions establish coding conventions. - Security scanners enforce vulnerability policies. - Merge requests bring automation and human review together. - IDE- or terminal-based agents generally see only the files provided to them, while the platform sees the full lifecycle, including deployment targets and approval rules. - Consequently, the platform—not the agent alone—largely determines what can ship safely. ## Security Implications of Faster Code Generation - Agents produce code and remediation patches faster, increasing both the number of vulnerabilities and the number of security fix merge requests. - The bottleneck shifts from finding vulnerabilities to deciding which AI-generated fixes should be reviewed and approved first. - Effective prioritization requires broader context, including: - Application data flows - Deployment environments - Project-wide code - Organization-wide security policies - GitLab’s security layer can filter false positives, confirm vulnerabilities, and use surrounding repository context to propose fixes. - Agentic SAST vulnerability resolution creates a merge request with the proposed remediation. - CI/CD validates the change, while a human reviewer retains final approval. - Strong quality gates and security controls applied within merge requests reduce the chance that vulnerabilities reach production. ## Repository Instructions with `AGENTS.md` - The tutorials use `AGENTS.md` to provide project-specific instructions. - These files can describe: - Repository structure - Commands to run - Code-quality expectations - Files or areas that must not be changed - Language and framework conventions - CI image-pinning requirements - Concurrency patterns - Custom instructions help agents operate consistently with project standards, even before they receive broader platform context. Organizations should connect coding agents to the systems that define requirements, quality, security, and approval. Repository access may enable a working patch, but lifecycle context is what makes that patch trustworthy and ready to merge.

Code Review CI/CD Gitlab Model Context Protocol+3

githubMay 22, 20263 min readCurated summary

GitHub recognized as a Leader in the Gartner® Magic Quadrant™ for Enterprise AI Coding Agents for the third year in a row

GitHub argues that AI coding has made code generation easier, shifting the main bottleneck to reviewing, securing, governing, and deploying software. It presents GitHub Copilot as an agentic platform spanning the full software development lifecycle, enabling developers to assign issues to agents and focus on reviewing and approving results. Gartner named GitHub a Leader in the 2026 Magic Quadrant for Enterprise AI Coding Agents, placing it highest for ability to execute for the third consecutive year. ## The Shift from Code Generation to Software Delivery - AI coding agents are increasingly expected to handle more than writing functions. - The harder problems now involve: - Code review - Security - Governance - Testing - Deployment - GitHub describes the new workflow as “orchestrating outcomes”: developers assign work to agents, then return to steer, review, and approve it. - Gartner projects that asynchronous AI coding-agent workflows could improve engineering productivity by 30%–50% by 2028, compared with 0%–20% gains from code assistants in 2025. ## Enterprise Adoption of GitHub Copilot - Copilot is used by 140,000 organizations, nearly three times the number reported a year earlier. - Overall growth exceeded 100% year over year. - Most users work with multiple AI models. - GitHub Copilot CLI usage nearly doubled month over month. - GitHub says these figures indicate that enterprises are adopting increasingly sophisticated, agent-driven workflows. ## Gartner’s 2026 Evaluation - Gartner evaluated 12 enterprise AI coding-agent vendors according to: - Ability to execute - Completeness of vision - GitHub was positioned as a Leader and ranked highest in ability to execute. - Gartner describes Leaders as vendors combining strong execution, market-shaping vision, rapid innovation, broad software-engineering relevance, and enterprise-grade security and governance. - The report’s Leader quadrant also includes Anthropic, Cursor, and OpenAI. ## GitHub’s Claimed Differentiators - **Developer choice:** Copilot supports multiple models and providers. - **Broad availability:** It works across editors, IDEs, CLIs, and GitHub’s web, desktop, and mobile applications. - **Full-lifecycle integration:** Copilot operates across issues, pull requests, code reviews, and GitHub Actions—not only inside the editor. - **Enterprise governance:** Teams can observe, audit, and secure how AI is used in engineering workflows. ## What GitHub Plans to Build Next - GitHub says it will expand agentic workflows across more developer-facing surfaces. - Planned investments include: - Greater model choice and intelligent model routing - Deeper integrations throughout the software lifecycle - Performance improvements based on how software is actually built and maintained on GitHub GitHub’s central recommendation is to treat AI coding agents as part of an end-to-end engineering platform rather than isolated code-generation tools. The post also notes that Gartner’s recognition is not an endorsement and that its findings should be considered alongside the full research report.

Code Review Github Copilot Github Copilot Cli Agentic Workflows+4