CI/CD

102 posts

gitlab3 min readCurated summary

Shai-Hulud copycat campaign targets Python developers through PyPI typosquatting

GitLab researchers uncovered a coordinated PyPI supply-chain campaign distributing a copy of the Shai-Hulud worm. Five packages—four typosquats and one compromised legitimate project—execute malware during Python startup, steal credentials from CI/CD and cloud environments, and propagate through developers’ repositories and package registries. The campaign demonstrates that Python packages can be weaponized without imports or explicit function calls. ## Malicious PyPI Packages - All packages were published by the `elitexp` account: - `rlask` and `tlask`, typosquats of Flask - `rsquests`, a typosquat of Requests - `nhmpy`, a typosquat of NumPy - `mflux-streamlit`, a legitimate project later weaponized in versions `0.0.3` and `0.0.4` - The attacker first uploaded clean probe versions matching current upstream version numbers, then replaced them with payload-bearing releases. - The activity followed the public release of Shai-Hulud’s source code, suggesting an independent copycat operation targeting Python users. ## Python Startup-Based Infection - The malware uses Python `.pth` files, which Python processes automatically at startup. - The dropper: - Checks for a `.bun_ran` marker in the temporary directory. - Downloads the Bun JavaScript runtime from GitHub. - Executes a roughly 5 MB obfuscated JavaScript payload. - Early `rlask` versions also included `sitecustomize.py`, which searched `sys.path` for and executed a hidden `_index.js` file. - This approach requires no explicit package import or function invocation. ## Payload Obfuscation - The JavaScript is protected by multiple layers: - Package-specific ROT-N encoding - AES-128-GCM encryption - Variable-name mangling using `_0x` identifiers - Researchers identified: - A small encrypted Bun downloader - A 772 KB Shai-Hulud credential stealer - Approximately 2,538 hardcoded strings ## Credential Theft The worm targets credentials and secrets from: - GitHub Actions tokens, repository secrets, OIDC tokens, artifacts, and runner memory - AWS IAM credentials, instance metadata, Secrets Manager, SSM, and STS tokens - Azure managed identities, Key Vault, and Microsoft Graph tokens - GCP service-account keys and application credentials - HashiCorp Vault tokens and Kubernetes authentication - npm, JFrog, PyPI, and RubyGems publishing credentials - SSH private keys and Kubernetes service-account tokens - Sigstore credentials and Fulcio signing certificates - MongoDB, MySQL, PostgreSQL, and Redis connection strings ## Self-Propagation Using stolen credentials, the worm can: - Add `.github/setup.js` and workflow files to repositories so it runs in other CI pipelines. - Insert `.github/copilot-instructions.md` to influence AI coding assistants. - Publish poisoned packages to PyPI, npm, and RubyGems. - Attempt privilege escalation on self-hosted runners through `sudoers` modifications. - Detect StepSecurity’s harden-runner and alter its behavior. ## Attacker Infrastructure and Weaponized Project - The PyPI account was created in 2024 and was associated with the legitimate `mflux-streamlit` project. - Package uploads used `Bun/1.3.14`, matching the runtime downloaded by the malware. - Unlike a pure typosquatting campaign, the compromise of a real project could affect existing users through normal dependency updates. Developers should audit environments for the affected packages, review CI/CD and cloud credentials, rotate exposed secrets, and enforce dependency pinning and package provenance checks. CI runners and publishing tokens should be treated as potentially compromised if any affected version was installed.

Read original(opens in new tab)
gitlab2 min readCurated summary

Mythos-class Claude Fable 5 arrives on GitLab Duo Agent Platform

Claude Fable 5, Anthropic’s Mythos-class model, is now available through GitLab Duo Agent Platform across all tiers and deployment models. The post presents it as a major step beyond incremental model improvements, emphasizing stronger first-attempt accuracy, sustained autonomous workflows, and better code review and incident analysis. Its main conclusion is that teams should apply it to complex, previously difficult engineering problems rather than routine tasks. ## More Accurate First-Pass Results - Produces more correct implementations for complex, well-defined problems with fewer iterations. - Reduces back-and-forth in Duo Agentic Chat. - Performs especially well on: - Multi-file refactoring - Incident investigation - Infrastructure-as-code definitions - Interprets technical images, web applications, and detailed screenshots more accurately while often using fewer output tokens. ## Longer and More Reliable Agent Workflows - Sustains goal-directed work across extended, multi-day runs and millions of tokens. - Maintains instructions and task focus without frequent manual checkpoints or re-prompting. - Uses verification loops to identify and correct its own mistakes. - Handles parallel sub-agents more reliably across multiple repositories or services. - Reduces the amount of human oversight required per agent run, allowing teams to review results asynchronously. ## Improved Bug Detection and Incident Response - Provides higher bug-finding recall than previous models. - Improves outage triage, root-cause analysis, and repository-history investigation. - Finds deeper code-path issues and edge cases during merge request reviews. - Produces more actionable review comments instead of generic feedback. - Helps reduce production defects and improve mean time to resolution. ## Recommended Use Cases - The post recommends testing the model on difficult, unsolved engineering problems rather than routine work. - Suggested applications include: - Complex multi-file refactors - Production incident investigations - Implementations developers would normally write manually - Teams should allow agents to scope work, ask clarifying questions, and execute multi-step solutions. Claude Fable 5 became available on GitLab Duo Agent Platform on June 9, 2026. Teams can access it through a free trial, the GitLab free tier, or included GitLab Credits for Premium and Ultimate subscribers.

Read original(opens in new tab)
cloudflare3 min readCurated summary

Your AI bill is out of control. Cloudflare can fix it now.

Cloudflare argues that uncontrolled AI spending stems from shared API keys, poor attribution, and defaulting to the most expensive models. Its AI Gateway now adds dollar-based spend limits, while a closed beta combines Cloudflare Access with identity-based budgets and routing. The goal is to make AI costs visible, attributable, and controllable without disrupting workflows. ### The AI Spending Problem - Companies encouraged aggressive AI adoption before establishing budgets or monitoring. - Shared API keys make it difficult to identify whether costs came from employees, teams, training pipelines, or runaway CI jobs. - Without routing rules or spending limits, users often choose frontier models for tasks that could use cheaper alternatives. - Visibility and controls are necessary to measure AI ROI and prevent waste. ### What AI Gateway Provides AI Gateway sits between applications and providers such as OpenAI, Anthropic, and Google. - Unified billing across providers and models. - Centralized logging of requests, token counts, and costs. - Response caching and rate limiting. - Content guardrails that can block PII and secrets before they reach models. - Previously, however, it lacked detailed attribution and budget controls. ### Dollar-Based Spend Limits - Budgets are measured in dollars rather than tokens and track cumulative request costs in real time. - Limits can be scoped by: - Model - Provider - Custom attributes such as user, team, or application - Budgets support daily, weekly, or monthly windows, including fixed and rolling periods. - When a limit is reached, requests can be blocked by default. - Dynamic Routes can instead redirect requests to a cheaper fallback model. - Spend limits are available in open beta across all AI Gateway plans through the dashboard or API. ### Identity-Driven Attribution and Policies Cloudflare Access can attach verified identity information to AI Gateway requests. - Authenticated user identities are extracted from Access JWTs and added as request metadata. - Organizations can track usage by employee, team, identity-provider group, or service. - Per-user budgets can give different spending limits to individual contributors and senior engineers. - Team policies can control which models different groups may use. - Access service tokens provide named identities for CI/CD pipelines and autonomous agents. - Logs include details such as email address, IdP group, or service-token name for export to analytics systems. - Configuration uses Cloudflare Access applications and existing identity-provider groups, without requiring custom Workers or manual JWT parsing. ### Cloudflare’s Internal Use - Cloudflare routes millions of requests and billions of tokens through AI Gateway each month. - It uses identity metadata to attribute costs, understand team usage, and manage budgets. - The company is making this internal approach available through the closed beta for identity-driven budgets and policies. Organizations should route AI traffic through a gateway, assign verified identities, set dollar-based budgets, and use model fallback rules to balance cost control with developer productivity.

Read original(opens in new tab)
meta3 min readCurated summary

Lights Out, Systems On: Validating Instant Power Loss Readiness

Meta’s Instantaneous PowerLoss Storm is a disaster-readiness testing program designed to prepare data centers for sudden, zero-notice power loss. It extends existing fault-tolerance mechanisms across facilities, servers, storage, compute, and the Twine orchestrator, while addressing region-wide failures and autonomous recovery. Through incremental testing and carefully defined tradeoffs, Meta aims to make losing an entire region as manageable as losing a smaller fault domain. ## Defense-in-Depth for Instant Failures - Power-loss tolerance was built into the full data-center stack, including mechanical and electrical systems, server racks, storage, compute, and Twine. - Batteries and Power Loss Siren (PLS) preserve in-memory data when racks lose power. - Twine services use region-wide asynchronous unavailability events (UEs) to coordinate shutdown and recovery. - Existing mechanisms had been tested against smaller fault domains, but region-wide failures introduced new challenges involving scale, replica placement, and autonomous startup. ## Solving Region Bootstrap Problems - Restarting a region may require millions of services to start simultaneously and discover their dependencies. - Circular dependencies among Twine control-plane services—such as Scheduler, Allocator, Broker, and Zelos—could prevent the orchestrator from starting itself. - Belljar CI/CD tests continuously identify critical startup dependencies before deployment. - A Twine recovery kit, supported by Belljar and Twrko, provides a manual “jumpstart” mechanism for breaking unexpected dependency cycles. - Meta also encountered a “boomerang” problem in which UEs shut down the control-plane services responsible for generating and distributing those signals. - The simpler solution was to let control-plane services ignore power-related shutdown UEs, preventing orphaned services that could not be reaped or recovered. ## Balancing Reliability and Engineering Velocity - Absolute tolerance to instant power loss could require costly or overly complex infrastructure and might create false positives during normal operations. - Meta defined unacceptable impacts as: - Storage or database data loss - Permanent damage to data-center facilities - Sustained disruption beyond one region - The company accepted bounded risks such as transient service errors, limited rack failures, and temporary staleness in routing or region-availability information. - Issues were considered tolerable when they could be remediated after the incident within a reasonable mean time to respond (MTTR). ## Incremental Validation Through PowerLoss Storms - Because testing a full region carried significant risk, Meta validated readiness progressively: - Dependency tests in new and pre-production regions - Exercises in shadow regions that mirror production - Tests in small production regions - Full tests in large regions supporting storage, AI, and data-warehouse workloads - During a Storm, Meta injects a power-supply fault to immediately de-energize an entire region. - After a short, realistic MTTR, remedial drain actions isolate the region from global controllers and schedulers. - The tests avoid preemptive preparation so they accurately represent an unexpected power failure. - Repeated exercises train both systems and engineers to handle regional loss with the resilience normally expected from smaller fault domains. Meta’s approach is to expand disaster readiness gradually: define unacceptable consequences, build layered recovery mechanisms, test at increasing scale, and use each exercise to improve both architecture and operational practice.

Read original(opens in new tab)
dropbox3 min readCurated summary

Beyond code generation: rethinking engineering productivity in the age of AI agents

AI coding agents have increased Dropbox’s code production, but they have also exposed bottlenecks in review, testing, release coordination, and operations. The central argument is that engineering productivity must be measured by end-to-end product velocity and customer impact—not code or pull-request volume alone. Dropbox is responding with agent platforms, stronger workflow infrastructure, broader quality metrics, and training that helps engineers adapt to new responsibilities. ## From Copilots to Agents - Copilots assist with explanations, snippets, and questions within existing workflows. - Agents can take scoped tasks, inspect repositories, edit files, run tests, fix failures, and return changes for human review. - Engineers remain responsible for intent, architecture, quality, and release decisions. - Increased parallel work and code output create new pressure on: - Code review systems - CI and testing infrastructure - Validation workflows - Release processes - Production operations - More code and pull requests do not necessarily create more customer value. ## Nova as Dropbox’s Agent Platform - Nova runs AI coding agents in controlled environments with relevant codebase context and internal engineering practices. - Its value comes from the surrounding platform—safe execution, workflow integration, guardrails, and human review—not only from the underlying model. - Nova generates approximately one in twelve Dropbox pull requests. - It supports both feature work and maintenance tasks, including: - Migrations - Flaky-test remediation - Bug investigation - Dependency updates - Other high-effort engineering work - The intended workflow is structured: define the task, let the agent work within constraints, validate the result, and require human approval before production. ## Measuring Product Velocity and Impact - Pull-request throughput was useful when implementation speed was the main constraint, but it is insufficient as AI increases output. - Dropbox evaluates whether the broader engineering system can absorb additional work efficiently. - Its four-stage measurement model tracks: - **Fuel:** Whether engineers use AI tools - **Adoption:** How teams change their workflows - **Output:** Whether AI contributes to production work - **Impact:** Whether products reach customers faster and create greater value - Quality and trust metrics include review turnaround time, first-run test pass rate, defect ratio, and rework rate. - Productivity improvements must not come at the expense of reliability or customer trust. ## Evolving Engineering Workflows - Engineers increasingly focus on defining intent, framing problems, reviewing changes, and making architectural and quality decisions. - Adoption requires more than tools; Dropbox uses hands-on learning, hackathons, bootcamps, workflow examples, and peer-led training. - Teams should adopt agents at different speeds based on risk, context, and readiness. - The goal is not to automate every workflow, but to make agentic development safe, useful, measurable, and repeatable. ## Broader Lessons - AI does not remove bottlenecks; it shifts them downstream. - Organizations must invest in validation, orchestration, governance, workflow integration, and measurement alongside code generation. - Competitive advantage will come less from access to common foundation models and more from the systems built around them: internal context, safeguards, quality controls, and integrated engineering workflows. Dropbox’s experience suggests that companies should treat AI agents as a change to the entire software delivery system, not merely as faster coding tools. The strongest results will come from improving the full path from idea to validated production impact.

Read original(opens in new tab)
gitlab3 min readCurated summary

GitLab Patch Release: 19.0.1, 18.11.4, 18.10.7 | GitLab Docs

GitLab released patch versions 19.0.1, 18.11.4, and 18.10.7 on May 27, 2026, addressing multiple security vulnerabilities and bug fixes across CE and EE. The most serious issue, rated CVSS 8.2, affected Duo AI workflow identity resolution, while other fixes addressed authorization, data exposure, denial-of-service, and CI access issues. GitLab strongly recommends that self-managed installations upgrade immediately. ## Release Scope and Upgrade Guidance - Applies to GitLab Community Edition and Enterprise Edition. - GitLab.com is already patched; GitLab Dedicated customers do not need to act. - The releases include fixes for supported versions and should be applied as soon as possible. - GitLab publishes vulnerability details publicly 30 days after the release containing the fix. - Patch releases are generally issued on the second and fourth Wednesdays, with critical ad-hoc releases possible. ## Security Fixes - **CVE-2026-4868 — Duo AI workflow runners, EE** - CVSS 8.2. - Under certain conditions, an authenticated user could cause workflows to run under another user’s identity. - Affected versions were EE releases before 18.10.7, 18.11.4, and 19.0.1. - **CVE-2026-1402 — Wiki denial of service, CE/EE** - CVSS 6.5. - Insufficient validation could allow an authenticated user to trigger a denial-of-service condition. - **CVE-2026-6713 — GraphQL WorkItem API, CE/EE** - CVSS 5.3. - Incorrect authorization checks could allow unauthorized enumeration of private projects. - **CVE-2026-5296 — Duo Workflows API, EE** - CVSS 4.3. - Developer-level users could bypass restrictions on group-level foundational flows in certain configurations. - **CVE-2026-2601 — Operations, EE** - CVSS 4.3. - Developers could potentially access sensitive deployment data because of missing authorization checks. - **CVE-2026-8716 — Pipelines, CE/EE** - CVSS 4.3. - Incorrect name resolution could expose CI data from an unintended ref type. - **CVE-2026-2710 — Authentication endpoints, CE/EE** - CVSS 4.3. - Blocked project access tokens could continue accessing private resources in certain cases. ## Bug Fixes in 19.0.1 - Added write permissions for repositories in job-token fine-grained permissions. - Fixed broken trial calls to action on the self-managed GitLab Credits dashboard. - Updated API security remediation guidance. - Removed Helm-based release-environment QA coverage. - Backported final 19.0 release notes and related maintenance changes. ## Bug Fixes in 18.11.4 - Updated Ruby thread scheduler behavior and Zlib to version 3.2.3. - Upgraded GitLab Shell to 14.50.0 and Elasticsearch indexer to 5.14.7. - Fixed lost comments when moving wiki pages. - Improved subgroup provisioning, synchronization workers, pipeline cancellation, and build handling. - Corrected issues affecting swimlanes, epic boards, CI catalog filtering, and pipeline traces. - Improved advanced-search bulk indexing by using the primary database connection. - Added AI workflow scope support and endpoint handling for diagram proxies. - Included performance optimizations for license approval workflows. ## Recommended Action Administrators of affected self-managed GitLab installations should upgrade to 19.0.1, 18.11.4, or 18.10.7 immediately, using the appropriate deployment method such as Omnibus, source, or Helm.

Read original(opens in new tab)
gitlab3 min readCurated summary

Agentic coding is only as good as its context

Coding agents are only as effective as the context they can access. Repository-only agents may produce compiling fixes, but they often miss issue requirements, CI rules, security policies, and review standards. Integrating agents with GitLab’s issues, merge requests, pipelines, and security controls produces better-aligned code, fewer review cycles, and safer releases while preserving human approval. ## From Repository Context to Full Lifecycle Context - GitLab tutorials demonstrate progressively richer workflows using Claude Code and Codex. - **Repository only:** The agent reads local files, interprets the prompt, and runs the build, but lacks acceptance criteria, organizational requirements, and review expectations. - **Repository plus issue:** Through GitLab’s MCP server, the agent can read issue descriptions, labels, milestones, and implementation notes. - It can correctly link changes to issues, such as adding `Closes #32`. - Tools such as `get_issue` and `create_merge_request` connect implementation work to planned work. - **Working inside the merge request:** GitLab’s Code Review Flow provides feedback, which the agent can address by adding tests, improving documentation, and fixing validation gaps. - New commits automatically trigger CI/CD. - Human reviewers can inspect the result in the same merge request. - The tutorials report fewer review rounds and shorter merge times. ## Why Platform Visibility Matters - Platform teams determine: - Which agents and tools are permitted - How agent output is verified - Where human approval is required - Important context lives in the DevSecOps platform: - Issues define requirements. - CI/CD configuration defines quality standards. - Review instructions establish coding conventions. - Security scanners enforce vulnerability policies. - Merge requests bring automation and human review together. - IDE- or terminal-based agents generally see only the files provided to them, while the platform sees the full lifecycle, including deployment targets and approval rules. - Consequently, the platform—not the agent alone—largely determines what can ship safely. ## Security Implications of Faster Code Generation - Agents produce code and remediation patches faster, increasing both the number of vulnerabilities and the number of security fix merge requests. - The bottleneck shifts from finding vulnerabilities to deciding which AI-generated fixes should be reviewed and approved first. - Effective prioritization requires broader context, including: - Application data flows - Deployment environments - Project-wide code - Organization-wide security policies - GitLab’s security layer can filter false positives, confirm vulnerabilities, and use surrounding repository context to propose fixes. - Agentic SAST vulnerability resolution creates a merge request with the proposed remediation. - CI/CD validates the change, while a human reviewer retains final approval. - Strong quality gates and security controls applied within merge requests reduce the chance that vulnerabilities reach production. ## Repository Instructions with `AGENTS.md` - The tutorials use `AGENTS.md` to provide project-specific instructions. - These files can describe: - Repository structure - Commands to run - Code-quality expectations - Files or areas that must not be changed - Language and framework conventions - CI image-pinning requirements - Concurrency patterns - Custom instructions help agents operate consistently with project standards, even before they receive broader platform context. Organizations should connect coding agents to the systems that define requirements, quality, security, and approval. Repository access may enable a working patch, but lifecycle context is what makes that patch trustworthy and ready to merge.

Read original(opens in new tab)
gitlab3 min readCurated summary

Full security scanner coverage of your codebase in minutes

GitLab 19.0 introduces security configuration profiles, enabling teams to centrally apply SAST, dependency scanning, and secret detection across many projects without editing individual `.gitlab-ci.yml` files. Profiles address coverage gaps caused by organizational growth, inconsistent configuration, and rapidly increasing AI-driven development. By applying default profiles in bulk, teams can achieve broad scanner coverage within minutes. ## Why Manual Scanner Configuration Falls Short - Per-project YAML configuration becomes difficult to maintain as organizations add repositories and teams. - Scanner settings can drift between frontend, backend, and older projects. - Pipeline changes may accidentally remove security scanners. - New projects may receive scanning while existing projects remain unprotected. ## Security Configuration Profiles - Profiles are centralized group-level settings defining how and when scanners run. - Teams can apply one profile to many projects through the GitLab UI. - GitLab provides default profiles for: - Static application security testing (SAST) - Dependency scanning - Secret detection - Default profiles use recommended settings and require no manual YAML changes. ## Scan Triggers and Coverage - SAST and dependency scanning run on: - Merge request pipelines, identifying vulnerabilities introduced by the proposed changes. - Pipelines on the default branch, maintaining a complete view of its security posture. - Secret detection supports both pipeline triggers plus push protection. - Push protection detects and blocks secrets during `git push`, before they enter the repository. - Push protection is event-based and therefore does not have a scan date in the security inventory. ## Practical Security Use Cases - Security teams can select hundreds of projects from the security inventory and apply scanners in bulk. - SAST can catch insecure code patterns before a merge request is approved. - Dependency scanning can detect compromised packages before they reach production. - Secret detection can block accidentally committed API keys immediately, avoiding later remediation and credential rotation. ## Getting Started - Available on GitLab Ultimate for GitLab.com, Self-Managed, and Dedicated. - Navigate to **Secure > Security inventory** for a group. - Select projects, choose **Manage security scanners** from **Bulk Action**, and select **Apply default profile to all**. - Review scanner status in the **Tool Coverage** column: - Green indicates full activation. - Partial coverage indicates some triggers are enabled. - Gray indicates the scanner is not configured. - Existing `.gitlab-ci.yml` settings can coexist with profiles, but inventory status may be inaccurate during the transition. Check the project’s **Security Configuration** page for the most reliable profile status. Organizations using GitLab Ultimate should apply default security profiles broadly, then review coverage and project-specific configurations to ensure every relevant trigger is active.

Read original(opens in new tab)
dropbox3 min readCurated summary

Introducing Nova, our internal platform for coding agents

Nova is Dropbox’s internal cloud platform for running coding agents across the software development lifecycle. Rather than building separate tools for coding, CI debugging, migrations, and operational tasks, Dropbox created a shared platform that supports interactive sessions and autonomous workflows within its monorepo and infrastructure. The platform grounds agent changes in real builds and tests, making AI assistance more reliable and easier to integrate into engineering workflows. ## The Case for a Shared Platform - Engineering work includes repetitive but important tasks such as: - Debugging CI failures - Updating dependencies - Improving test coverage - Fixing flaky tests - Managing migrations and operational work - Different tasks require different interaction models: - Interactive chat for developer-driven work - Asynchronous workflows for long-running remediation and automation - Dropbox’s environment has specialized requirements: - A large monorepo - Bazel for builds and tests - Caching and remote execution - On-premises infrastructure - Dropbox-specific validation workflows - Off-the-shelf coding agents were designed primarily for local development and did not naturally fit this environment. ## How Nova Runs Coding Sessions - Each session runs in an isolated environment using a specific snapshot of the codebase. - Callers provide: - The repository commit - A task description - Optional validation commands - Iteration limits and branch settings - Nova can run builds and tests after an agent proposes a change. - If validation fails, the results are sent back to the agent so it can continue troubleshooting. - This creates a feedback loop of: - Propose a change - Validate it in the real environment - Correct failures - Repeat as needed - Nova supports multiple coding agents behind a common interface. - Engineers can access it through: - A web interface - A command-line client - An API - Internal scripts and services - The platform also provides prompt evaluation, observability, feedback collection, skills, plugins, and MCP integrations for accessing systems such as logs and monitoring tools. ## Deterministic Code Publication - Nova keeps code publication outside the agent. - Each session is limited to a single branch. - This makes active work and publication status predictable. - It avoids the complexity of agents creating and managing multiple branches. - The deterministic model simplifies automation such as: - Running tests - Rebasing onto the main branch - Tracking which changes belong to each session ## Engineering Workflows Using Nova ### Developer-Driven Sessions - Engineers use Nova’s web interface for quick fixes and prototypes without disrupting local work. - Validation commands can use Bazel selectivity tools to target the relevant compile and test dependencies. - Slack discussions can be carried into Nova sessions, preserving context and reducing manual setup. ### Flaky Test Remediation - Dropbox built Deflaker, a durable workflow connected to Athena, its flaky-test detection system. - Deflaker gathers examples of a test passing and failing. - It sends the associated logs to Nova. - The agent analyzes the evidence, identifies a likely cause, and proposes a fix. - This demonstrates how Nova can combine investigation, context gathering, and code changes in a longer-running automated process. ## Practical Takeaway Dropbox’s experience suggests that coding agents are most useful when embedded in existing engineering systems rather than treated as isolated code-generation tools. A shared platform like Nova can support many workflows while preserving consistent execution, validation, context, and observability.

Read original(opens in new tab)
gitlab2 min readCurated summary

Transform MRs from manual tasks to an automated workflow

GitLab 19.0 expands Developer Flow from generating merge requests to managing much of their entire lifecycle. Its AI agent can respond to reviews, investigate codebases, resolve conflicts, and split oversized MRs, while automation handles rebasing and merging. The result is less manual effort between opening and merging an MR, with developers supervising rather than executing every step. ## Developer Flow Across the MR Lifecycle - Can be triggered from: - An issue via **Generate MR** - An issue or MR assigned to the **Duo Developer** service account - Any issue or MR discussion using the new **@mention** trigger - Continues working on the same MR instead of creating separate changes to reconcile. - Handles: - Multiple rounds of reviewer feedback - Merge conflicts on long-running branches - Codebase research and technical evaluations - Oversized MR splitting - New feature implementation - Uses a single agentic loop with tools such as `read`, `grep`, file editing, and command execution. - Reads `AGENTS.md` for project conventions and operational guidance. - Uses `agent-config.yml` to configure dependencies, tooling, tests, and pre-commit hooks. These capabilities are available through GitLab Duo Agent Platform on Premium and Ultimate plans. ## Autonomous Merge Conflict Resolution - The beta **Resolve with Duo** button is available on the MR conflict page and merge checks widget. - The agent: - Reviews the MR’s intent and both branches - Selects a resolution strategy - Edits conflicting files - Commits and pushes the resolution - It leaves a summary comment explaining the conflict and resolution path. - If it cannot resolve the conflict safely, it reports that rather than guessing. ## One-Click Rebase and Merge - The beta feature combines rebasing and merging into one action. - It is designed for teams using semi-linear or fast-forward merge methods. - It is available on Free, Premium, and Ultimate tiers. ## Reducing Manual MR Work GitLab distinguishes between AI-driven judgment and mechanical automation: - AI handles code changes, reviewer feedback, and conflict resolution. - Automation handles tasks such as rebasing before merge. - Together, these features reduce the time developers spend on repetitive MR maintenance while preserving human oversight for steering, reviewing, and final decisions. Developers can try Developer Flow through a GitLab Duo Agent Platform trial. Existing Premium and Ultimate users with the platform can use it on merge requests, while older GitLab versions may require manually configuring the mention trigger.

Read original(opens in new tab)
gitlab3 min readCurated summary

Manage CI/CD credentials with GitLab Secrets Manager

GitLab Secrets Manager, entering public beta with GitLab 19.0, provides a native way to manage CI/CD credentials without storing them in broadly scoped variables or configuration files. Built on OpenBao and integrated with GitLab’s existing permissions, environments, branches, and audit trails, it aims to reduce credential exposure and simplify incident response. The post recommends trying it in existing GitLab projects and pipelines, especially where least-privilege access is difficult to enforce. ## Where CI/CD Secrets Commonly Go Wrong - Developers often store credentials in: - Project- or group-level CI/CD variables - Configuration files - Accidentally committed `.env` files - Masked CI/CD variables may still be exposed to every job and anyone with pipeline access. - Standalone vaults improve separation but introduce: - A second authentication system - Separate permission models - Additional audit logs and operational overhead ## Using GitLab Secrets Manager - Secrets are stored within GitLab’s existing project and group structure. - Pipelines reference secrets with the `secrets:` keyword in `.gitlab-ci.yml`. - By default, GitLab writes the secret to a temporary file and exposes its path to the job. - Passing a file path instead of the raw value can reduce exposure in: - Subprocesses - Crash dumps - Telemetry systems ## GitLab-Based Access Controls - Secrets use GitLab’s existing users, groups, projects, and roles. - Permissions can be assigned for reading, creating, updating, and deleting secrets. - Group-level secrets are inherited by nested projects, allowing common credentials to be defined once. - Removing someone from a project or group immediately removes their access to its secrets. - This avoids maintaining a separate access hierarchy that could drift from GitLab’s permissions. ## Job-Level Secret Scoping - Each secret can be restricted based on: - Target environment - Branch - Whether the branch is protected - Wildcards such as `production/*` simplify environment and branch rules. - Multiple conditions can be combined, such as requiring both a protected branch and a production environment. - At runtime, the backend verifies the job’s identity and scope before returning the secret. - Secrets are discarded when the job ends, and job logs are masked. - Narrow scopes reduce the systems affected if a dependency or pipeline is compromised. ## Auditing Secret Usage - Secret creation, updates, and deletions appear in GitLab’s existing audit trail. - Pipeline secret reads include the originating pipeline and job IDs. - Responders can trace where a credential was used without correlating separate systems manually. - Audit logging is available for self-managed deployments; GitLab.com support is expected during the beta. ## Public Beta Availability - The beta is available to Premium and Ultimate users on GitLab.com and self-managed deployments. - GitLab Dedicated support is planned. - The feature is free during beta and will later become a paid GitLab Credits feature. - Existing integrations with HashiCorp Vault, AWS Secrets Manager, Azure Key Vault, and Google Cloud Secret Manager remain available. GitLab Secrets Manager is best suited for teams that want least-privilege CI/CD credentials while keeping access control and auditing within GitLab. Teams can adopt it incrementally alongside existing external secrets platforms.

Read original(opens in new tab)
gitlab4 min readCurated summary

GitLab 19.0 | GitLab Docs

GitLab 19.0, released May 21, 2026, expands AI-assisted development, work-item customization, secrets management, and dependency security. Major changes include group-level Duo review instructions, configurable work-item types, open beta access to GitLab Secrets Manager, and generally available SBOM-based dependency scanning. GitLab also introduces usage-based billing for Duo Core and adds new agent, search, model, and workflow capabilities. ## Customization and Project Management ### Group-level custom review instructions for GitLab Duo - Premium and Ultimate feature for GitLab.com, Self-Managed, and Dedicated. - Groups and subgroups can share review guidance through: ```text .gitlab/duo/mr-review-instructions.yaml ``` - A project in the group serves as the template. - Group instructions are combined with project-specific instructions. - Supported by both Code Review Flow and GitLab Duo Code Review. ### Configurable work item types - Projects can define custom types such as User Story, Bug, or Maintenance instead of using only Issues and Tasks. - Each type has its own name and icon. - Types support custom fields, status lifecycles, saved views, and issue boards. - Configuration at the top-level group or organization cascades to projects. - Administrators can enable or disable types globally or allow project-level control. - Disabling a type does not affect existing work items. ## Security and Dependency Management ### GitLab Secrets Manager enters open beta - Available to Premium and Ultimate customers on GitLab.com and GitLab Self-Managed. - Project and group Owners can store and reference CI/CD secrets in GitLab. - Secrets are scoped to projects or groups and available only to jobs that explicitly request them. - The feature remains subject to beta support policies and may not be production-ready. ### SBOM-based dependency scanning becomes generally available - Available for Ultimate customers across GitLab offerings. - Maven, Gradle, and Python projects receive visibility into transitive dependencies and their vulnerabilities. - Automatic dependency resolution runs when no lockfile or dependency graph is available. - If resolution is unavailable, manifest scanning examines direct dependencies in files such as: - `pom.xml` - `requirements.txt` - `build.gradle` - `build.gradle.kts` - Manifest scanning is enabled by default, while full transitive coverage requires dependency resolution, a lockfile, or a manually supplied dependency graph. ## GitLab Duo and Agentic Development ### Duo Developer enhancements - GitLab Duo Developer can be triggered by: - Assigning it to an issue - Selecting **Generate MR** - Mentioning it with `@mention` in an issue or merge request discussion - It can turn feedback, to-do items, and design questions into code changes, follow-up merge requests, or research summaries. - With `AGENTS.md` and `agent-config.yml`, it can run tests and checks before committing. - Administrators can enable mention and assignment triggers for eligible projects. ### Duo Core adopts usage-based billing - Code Suggestions in the Web IDE and desktop IDEs now consume GitLab Credits. - Duo Chat becomes agentic for Duo Core users and runs on the GitLab Duo Agent Platform. - Administrators must enable the Agent Platform for the instance or top-level group to use Chat in GitLab or desktop IDEs. ### New agent and search capabilities - Exact code search supports repository filtering with the `repo:` syntax: ```text def authenticate repo:my-group/my-project ``` - Flows and external agents can trigger when a draft merge request is marked ready for review. - The merge request ready trigger is controlled by the `merge_request_ready_flow_trigger` feature flag and is disabled by default. - Claude Opus 4.7 is available in the Duo Agent Platform for complex, multistep tasks involving code review, CI/CD, and vulnerability resolution. - GitLab Duo Agent Platform Self-Hosted adds compatibility with Gemini models and supports multiple flows, including Code Review Flow and SAST vulnerability workflows. GitLab 19.0 is particularly significant for teams adopting AI agents and centralized development governance. Organizations should review Duo’s new billing model, test Secrets Manager carefully during its beta period, and enable dependency resolution to obtain comprehensive vulnerability coverage.

Read original(opens in new tab)
gitlab2 min readCurated summary

Track CI component usage across your organization

GitLab 19.0 adds Components Analytics to the CI/CD Catalog, giving organizations visibility into how shared CI/CD components are adopted and which versions projects use. All tiers provide high-level usage counts, while GitLab Ultimate offers project-level version tracking and outdated-version identification. The feature helps platform teams respond to security issues, manage upgrades, and govern increasingly AI-generated pipelines. ## The Visibility Gap in Shared CI - The CI/CD Catalog centralizes versioned, reusable pipeline components. - Previously, maintainers could not easily determine: - Whether components were being used - Which projects depended on them - Which versions were still active - Security fixes did not automatically reach projects using older component versions, making organizational exposure difficult to measure. ## High-Level Adoption Analytics - Available across all GitLab tiers, including Free; introduced in GitLab 18.9. - Found under **Explore > CI/CD Catalog > Analytics**. - Shows, for each maintained catalog resource: - The latest released version - The number of unique projects using it in the past 30 days - Components available in that version - Helps teams prioritize maintenance, plan deprecations, and assess investment in shared CI infrastructure. ## Component Usage Detail in Ultimate - GitLab Ultimate provides per-component drill-down analytics. - Maintainers can see: - Which projects used each component in the past 30 days - The version used by each project - Whether each project is up to date or outdated - This makes it easier to respond to vulnerabilities, notify project owners, open merge requests, and assess the impact of refactors or deprecations. ## Native Governance Compared with Other Platforms - GitHub Actions lacks native organization-wide catalog analytics for reusable workflows. - CircleCI Insights focuses on pipeline performance rather than orb adoption and versions. - Jenkins Shared Libraries require custom tooling to track usage. - GitLab combines a governed component catalog with built-in adoption and version visibility. ## Supporting AI-Generated Pipelines - The catalog establishes standardized CI practices, while analytics verifies whether those standards are actually used. - This is increasingly important as AI tools generate more production pipelines. - Self-Managed and Dedicated customers can mirror GitLab components and combine them with internally built components for regulated or air-gapped environments. Organizations maintaining CI/CD Catalog components can use adoption metrics immediately. Teams needing project-level version and remediation details require GitLab Ultimate.

Read original(opens in new tab)
gitlab2 min readCurated summary

Beyond BYOK: Why governance matters for AI agents

BYOK and local models give developers more control over which AI systems they use, but they do not provide enterprise governance by themselves. The post argues that AI agents operating in CI/CD need platform-level controls for authorization, security, and auditing, especially when no human is present. It presents GitLab Duo CLI and its Agent Platform as a governance-oriented alternative for controlled, auditable automation. ## Terminal AI: Individual Tool vs. Platform - Copilot’s BYOK and local-model support primarily extend AI capabilities at an individual developer’s workstation. - The post argues these features do not enforce organization-wide model policies or provide a complete audit trail of agent actions. - GitLab Duo CLI is positioned for both interactive development and automated workflows across multiple projects and release cycles. - Its headless mode allows it to run non-interactively and scriptably inside CI/CD pipelines. ## Why Model Choice Is Not Governance - Interactive coding tools generally assume a human reviews every action. - Automated agents can run tests, modify configuration, and perform multi-step delivery tasks without continuous oversight. - Enterprise governance therefore requires answers to questions such as: - What resources can the agent access? - Which actions is it authorized to perform? - Can the organization prove what the agent did? - GitLab’s platform-level controls include: - Human approval for actions in interactive mode. - Prompt-injection detection. - Composite identity scopes limiting agent access. - `AGENTS.md` and `SKILL.md` files for defining permitted tasks and actions. ## CI/CD Automation Requires Consistent Controls - Potential use cases include debugging failed pipelines and completing multi-step development work. - Pipeline-based agents cannot rely on a developer to detect prompt injection or unexpected behavior. - Security controls must therefore be built into the platform and applied consistently across workflows and environments. ## Model Flexibility and Data Sovereignty - The post recommends evaluating whether AI tooling maintains its security model when no human is watching. - GitLab Duo CLI supports both self-hosted and GitLab-hosted models. - Organizations can keep sensitive workloads on infrastructure they control while using hosted models for other tasks. ## Practical Recommendation Model flexibility is useful, but production adoption depends on governance. Teams considering AI agents for CI/CD should prioritize authorization, auditing, prompt-injection protection, and consistent platform controls—not just BYOK or offline execution.

Read original(opens in new tab)
spotify3 min readCurated summary

Coding Is No Longer the Constraint: Scaling Developer Experience to Teams and Agents at Spotify | Spotify Engineering

Spotify argues that AI has shifted software development’s main constraint from writing code to coordinating people, systems, and decisions. Years of investment in standardized platforms, automation, and developer experience enabled Spotify to adopt AI coding tools at extraordinary scale. The company’s experience suggests that consistent infrastructure and strong feedback loops are essential for making both human developers and coding agents effective. ## Rapid AI Adoption - More than 99% of Spotify engineers use AI coding tools weekly. - 94% report improved productivity. - Pull request frequency has increased by 76%, with most PRs created by developers working alongside AI agents. - Adoption accelerated sharply after the release of Claude Opus 4.5. ## Fleet Management Before AI Agents - Spotify’s codebase was growing seven times faster than its engineering workforce. - Developers increasingly spent time on dependency upgrades, API migrations, and vulnerability fixes. - Fleet Management automated changes across hundreds or thousands of components. - Its orchestration system, Fleetshift, has merged more than 2.5 million maintenance PRs, most without human intervention. - This approach reduced migrations from work taking weeks or months across many teams to centrally managed operations. ## Honk: A Background Coding Agent - Deterministic scripts struggled with complex refactoring and the edge cases found across large codebases. - Spotify created Honk, a background coding agent powered by Claude through the Agent SDK. - Honk runs in Kubernetes pods, allowing many coding sessions to execute concurrently. - It can use trusted tools and run builds in CI across multiple operating systems. - Fleetshift identifies targets, schedules work, and tracks PRs, while Honk performs the code changes. - A recent Java migration across Spotify’s backend services took three days. - Engineers can invoke Honk through Slack, where it uses conversation context to create and return PRs. - Honk v2 adds shared sessions, team projects, and agent orchestration through Chirp. ## Standardization Improves Agent Performance - Spotify’s principle of limiting the number of technologies it supports reduces decisions and improves collaboration. - Consistent service architectures and design patterns also give AI agents better reference material. - Agents perform worse in fragmented codebases with inconsistent conventions. - Backstage provides a unified internal developer portal and catalog for software components. - Spotify exposes Backstage capabilities to agents through MCP integrations and command-line tools. - Agents can discover component ownership, read documentation, and contact responsible teams. ## Guardrails Through Backstage - Backstage’s Soundcheck and “golden state” define recommended technologies and practices. - Teams can assess their components against these standards. - Static analysis and linting provide immediate feedback when developers or agents use unsuitable patterns. - This creates a feedback loop that helps agents correct their work and drives consistency across the organization. Spotify’s experience indicates that scaling AI development requires more than giving engineers access to models. Organizations should invest in standardized platforms, searchable component metadata, automated fleet-wide workflows, and strong validation systems so agents can operate reliably at team scale.

Read original(opens in new tab)