How we built a software factory to drive Astro’s GitHub issue count to zero
AI-powered software factories can address a pressing open-source problem: maintainers are overwhelmed by the flood of AI-generated issues, pull requests, and security reports. The Astro team built an automated triage pipeline that reproduces bugs, diagnoses causes, creates fixes, and ships preview releases for verification. After several months, it reduced Astro’s open issues from more than 200 to roughly 30 without mass-closing or ignoring reports. ## Building an Issue-Triage Skill - The team began by automating issue triage, one of the most time-consuming parts of open-source maintenance. - The workflow mirrors manual debugging: - **Reproduce:** Clone the reporter’s reproduction repository and confirm the problem. - **Diagnose:** Instrument the code and add logging to identify the root cause. - **Verify:** Check tests, documentation, and comments to determine whether the behavior is actually a bug. - **Fix:** Turn the reproduction into failing tests, implement a solution, and deploy it. - Each phase runs in an isolated AI subagent to reduce the tendency to force a solution. - Subagents communicate through a sequential `report.md` file containing their findings. ## Running the Pipeline in GitHub Actions - The workflow is driven by GitHub issue labels rather than a separate internal database. - New issues begin with `triage needed`; verified fixes eventually move to `fix verified`. - The pipeline reconstructs its state from labels and existing issue comments. - When a fix is ready, it: - Creates a preview release using `pkg.pr.new`. - Posts the diagnosis, logs, and installation instructions to the issue. - Lets the original reporter test the patch. - Opens a linked pull request after confirmation. ## From a Repository Workflow to Flue - The team recognized that the process was not inherently tied to GitHub. - Its core structure consists of: - An external event. - A sequence of isolated subagents. - Separate reasoning and execution permissions. - Durable workflow state. - This generalization became **Flue**, an open, platform-agnostic framework for agent workflows that can respond to GitHub events, Slack messages, cron jobs, or webhooks. ## Effects on Maintainer and Community Work - Automation did not make the Astro team less connected to users. - Instead, it freed maintainers to spend more time: - Engaging with the community in Discord. - Participating in RFCs and feature discussions. - Collaborating with contributors. - The system is designed to resolve most incoming issues, while failures are treated as signals that the codebase needs improvement. ## Using Agent Failures to Improve the Codebase Agent mistakes often reveal problems that would also challenge human developers: - **Opaque abstractions:** Component boundaries are unclear. - **Missing documentation:** Important implementation decisions are unexplained. - **Insufficient testing:** Critical behavior lacks adequate unit tests. - For example, the bot repeatedly changed an HMR-related condition and caused regressions because the logic was poorly documented and under-tested. - Adding a precise comment clarified the intended behavior, after which the bot stopped making the same incorrect change. - Fixing these weaknesses improves both future automation and human maintainability. ## Extracting the Workflow into a GitHub Action - Initially, the triage system was embedded in the Astro monorepo, making changes risky and difficult to test. - The team separated it into the standalone `triagebot-action` repository. - This enabled independent testing and safer updates to Flue and the workflow. - The action now supports Astro and has been adopted or forked by other teams building their own automated development pipelines. The practical lesson is to start with a narrow, repeatable maintenance task, isolate agent responsibilities, make all reasoning auditable, and use failures to improve documentation, architecture, and tests.
Read original(opens in new tab)
