Model Context Protocol - GitHub

githubAug 11, 20262 min readCurated summary

From coder to orchestrator: How agents shift the role of a developer

AI agents can generate impressive one-prompt demos, but reliable software delivery requires more than isolated outputs. Developers increasingly need to design workflows that define how code is proposed, tested, reviewed, and shipped. The article argues that this shifts developers from primarily writing code to orchestrating agents within controlled, repeatable systems. ## From One-Off Prompts to Reliable Workflows - A single prompt can quickly produce a demo, such as a simple game. - Production development requires repeatable delivery with: - Appropriate context - Validation and testing - Security controls - Review processes - Clear permissions and handoffs - GitHub Copilot is presented as a control plane for connecting these parts. ## An Agentic Development Flow - Familiar repository events can trigger agent work, including: - Adding a label to an issue - Running a scheduled workflow - Starting a GitHub Actions process - The agent’s changes are captured in a pull request. - Deterministic checks then validate the work through: - Linting - Tests - Security scans - Build verification - CODEOWNERS, required reviews, and branch protection rules control what can be merged. - Agents handle ambiguous, context-heavy tasks, while predictable automation provides the safety boundary. - Developers decide: - What agents can access - How tasks are scoped - Where workflows hand off - When human judgment is required ## GitHub’s Implementation Options - Copilot cloud agent workflows support event-driven automations. - Copilot CLI can run AI-powered steps inside GitHub Actions. - Model Context Protocol (MCP) can extend agents with additional tools and external context. - These options represent different stages of building an agent-enabled development workflow. ## Starting Small - Teams should begin with one bounded, low-risk workflow. - Suitable examples include: - Issue triage - Synchronizing documentation and tests - Routine maintenance updates - The recommended approach is to integrate Copilot into existing development infrastructure rather than redesigning everything at once. Developers should treat AI agents as components within an engineered delivery system, not as replacements for that system. Start with a limited workflow, surround agent output with automated checks and review controls, and gradually expand as the process proves reliable.

Read original

Model Context Protocol AI Agents CI/CD Github Copilot+4

githubJun 25, 20263 min readCurated summary

Evaluating performance and efficiency of the GitHub Copilot agentic harness across models and tasks

GitHub argues that an agentic harness—the system coordinating tools, context, and workflow—strongly influences how effectively an AI model solves software tasks. Its shared Copilot harness aims to match model-vendor harnesses in task completion while using fewer tokens. Benchmark results across four models suggest near-parity overall, with performance varying by model and task. ## Benchmarking Approach - GitHub evaluates the harness using public and internal benchmarks, plus real-world metrics and online experiments. - Comparisons hold the following constant: - The same underlying model - The same task - Context-window size - Reasoning effort - Tool selection - MCP servers - Copilot CLI is compared with: - Claude Code for Claude Sonnet 4.6 and Claude Opus 4.7 - Codex CLI for GPT-5.4 and GPT-5.5 - Benchmarks include: - **SWE-bench Verified:** 500 human-validated Python bug fixes - **SWE-bench Pro:** More complex, multi-step engineering tasks - **SkillsBench:** Skill usage and triggering - **TerminalBench:** Command-line workflows - **Win-Hill:** Software tasks in Windows containers ## Token Efficiency - Copilot’s harness generally completes tasks at rates comparable to model-vendor harnesses while consuming fewer tokens. - It performed better across the reported tests for Sonnet 4.6 and Opus 4.7. - For GPT-5.4 and GPT-5.5, Copilot performed better on nearly every benchmark. - The main exception was SWE-bench Verified, where Copilot was 7% worse with GPT-5.4 and 4% worse with GPT-5.5. ## Task Resolution - GitHub emphasizes that lower token use matters only when the agent successfully completes the work. - Overall task-resolution rates were considered on par with vendor harnesses. - Results varied by benchmark: - Copilot generally did better on SWE-bench Pro, especially with GPT models. - It outperformed vendor harnesses on Win-Hill or matched them for every model. - It performed better on TerminalBench with Sonnet and Opus, matched GPT-5.5, and trailed GPT-5.4. - SkillsBench results favored GPT models on Copilot but favored vendor harnesses for Claude models. - Differences are described as statistically comparable because model behavior is stochastic and run-to-run variation can explain many gaps. ## TerminalBench Variance Analysis - GitHub uses TerminalBench 2.0 to study both cost and completion rate over repeated runs. - The preferred outcome is higher resolution with lower cost. - The analysis illustrates that benchmark results should account for variance rather than relying on a single run. - GitHub presents Copilot as equal to or better than the vendor harnesses on this cost-versus-success comparison. The practical conclusion is that harness design is a reusable performance multiplier across Copilot products. GitHub’s results support using its shared harness when developers need broad model choice, efficient token usage, and comparable task-completion performance, while recognizing that the best harness can still vary by model and workload.

Read original

Model Context Protocol Large Language Models AI Agents Github Copilot+3

githubJun 19, 20263 min readCurated summary

How we built an internal data analytics agent

Qubot is GitHub’s internal, Copilot-powered analytics agent, designed to make warehouse data accessible without requiring an analyst. Employees ask natural-language questions through Slack, VS Code, or the Copilot CLI, while Qubot uses curated context and selects between Kusto and Trino to produce answers. GitHub’s experience shows that structured context is central to improving both accuracy and response speed. ## The Problem Qubot Addresses - Product teams often struggle to identify the right data model, grain, filters, and query. - Limited analyst availability leaves many teams to investigate telemetry independently. - Qubot targets exploratory questions rather than replacing dashboards or formal reporting. - It enables employees to investigate unfamiliar datasets with minimal setup and maintenance. ## Multiple Ways to Ask Questions - **Slack:** Users ask questions in a shared channel, receive answers in threads, and collaborate on follow-up questions. - Each result is saved as a Markdown report in a pull request, allowing users to refine queries or reuse them in dashboards. - **VS Code and Copilot CLI:** Qubot is installed as a plugin and operates alongside other agents, skills, and tools. - Offering both zero-configuration and developer-focused interfaces increased adoption among different user groups. ## A Federated Context Layer Qubot’s context is tailored to GitHub’s warehouse layers: - **Bronze:** Product teams provide telemetry schemas and metadata for raw events. - **Silver:** Data teams maintain query examples, usage guidance, and required filters for conformed data. - **Gold:** Dataset owners contribute business rules and metric definitions. - ETL pipelines add derived metadata and other signals automatically. - Context is fetched at runtime through the GitHub MCP Server. A dedicated context agent standardizes contributions from different teams. It ingests Markdown documentation and repository references, then organizes and normalizes them into a structure that Qubot can use effectively. ## Evaluation Before Deployment Every change to Qubot’s instructions or context layer is tested through an offline evaluation framework. - Test cases contain prompts, expected answers, ground-truth SQL, domains, and difficulty levels. - Automated orchestration launches multiple parallel agent trials using `gh agent-task create`. - Results are saved as JSON and aggregated by completion rate, accuracy, and duration. - Teams compare configurations and detect regressions before releasing changes. ## Choosing Between Kusto and Trino Qubot connects to both query engines through MCP servers: - **Kusto** is used for fast exploration of recent event data. - **Trino** handles complex joins and deeper historical analysis. - Qubot defaults to Kusto and switches to Trino when the question requires more advanced or historical querying. - This hides query-engine selection from users. ## Adoption and Lessons Learned - Hundreds of GitHub employees have run thousands of queries through Qubot. - Routine questions in analytics support channels declined as teams gained more autonomy. - The agent helped employees who previously avoided the warehouse access data for decision-making. - GitHub found that well-structured, carefully maintained context improved accuracy and made Qubot approximately three times faster at finding the right answer. The main recommendation is to treat analytics context as a maintained engineering asset. A capable agent depends not only on model intelligence, but also on accurate metadata, documented business rules, evaluation-driven iteration, and interfaces that fit users’ existing workflows.

Read original

Model Context Protocol AI Agents Slack Github Copilot+4

githubJun 17, 20263 min readCurated summary

Getting more from each token: How Copilot improves context handling and model routing

GitHub is improving Copilot’s efficiency by optimizing both how context is handled and which model performs each task. Prompt caching and deferred tool loading reduce repeated overhead in long agentic sessions, while Auto selects models based on task complexity and real-time system conditions. The goal is to preserve quality while using tokens, compute, and cost more intelligently. ## Prompt Caching and Deferred Tools - Copilot sessions repeatedly include instructions, repository context, conversation history, tool definitions, and task state. - Prompt caching allows repeated prompt prefixes to reuse model state instead of being recomputed on every request. - Tool search loads tool definitions only when needed, rather than placing every full schema into context on every turn. - This is especially valuable for sessions involving MCP tools, terminal commands, file operations, workspace search, and other actions. - The approach reduces fixed per-turn overhead while preserving access to a broad toolset. ## Auto Model Selection - Copilot Auto chooses a model based on the task rather than requiring developers to select one manually. - Quick explanations, focused edits, and complex multi-file changes often benefit from different levels of reasoning. - Evaluations found that no single model consistently performs best across all tasks. - Auto uses more efficient models when they can achieve comparable results and selects stronger reasoning models when the task demands them. - The objective is not simply lower cost, but matching model capability to the work. ## How Auto Routes Tasks Auto combines two signals: - **Real-time model health** - Considers availability, utilization, response speed, error rates, and cost. - Avoids selecting a model that is technically capable but currently overloaded or unreliable. - **Task-aware routing with HyDRA** - Evaluates reasoning depth, code complexity, debugging difficulty, and tool-orchestration requirements. - Identifies models capable of meeting the task’s quality threshold, then chooses the best fit among them. - The system can be tuned toward maximum quality or greater cost savings. ## Cache-Aware Routing - Switching models during every turn can reduce efficiency by invalidating cached prompt prefixes. - Auto therefore keeps the same model during a conversation so the cache can continue building. - It changes models at natural cache boundaries: - The first turn, when no cache exists yet. - After context compaction, when older turns are summarized and the prompt prefix resets. ## Multilingual Routing - Copilot’s routing system was trained on conversations spanning 16 language families, including CJK and European languages. - Evaluations covered production VS Code chat data across 19 languages. - Routing accuracy remained within four points of the English baseline across language groups. - No statistically significant quality gap was observed between the language groups. Copilot’s efficiency strategy combines smarter context management with adaptive model selection. Caching and on-demand tools reduce repeated work, while cache-aware Auto routing chooses an appropriate model without unnecessarily sacrificing quality or session efficiency.

Read original

Model Context Protocol AI Agents Github Copilot Visual Studio Code+3

githubMay 7, 20264 min readCurated summary

Improving token efficiency in GitHub Agentic Workflows

GitHub’s Agentic Workflows can quietly accumulate substantial token costs because they run automatically in CI. GitHub improved efficiency by instrumenting token usage, auditing workflows, pruning unused MCP tools, and replacing many MCP data-fetching calls with deterministic GitHub CLI commands. Early results show that reducing context and removing unnecessary LLM reasoning can save thousands of tokens per run, though measuring true efficiency requires accounting for model choice and workload quality. ## Logging Token Usage - GitHub runs hundreds of agentic workflows against real GitHub Actions limits. - Different agent frameworks produced incompatible usage logs, so GitHub used its API proxy to normalize data across Claude CLI, Copilot CLI, and Codex CLI. - Each workflow now emits a `token-usage.jsonl` artifact containing: - Input, output, cache-read, and cache-write tokens - Model and provider - Timestamps - One record per API call - These records make it possible to compare historical runs and identify recurring sources of waste. ## Automated Auditing and Optimization - A daily **Token Usage Auditor** aggregates recent usage by workflow and reports: - Significant increases in token consumption - The most expensive workflows - Anomalous runs, such as a workflow taking 18 LLM turns instead of its usual four - A daily **Token Optimizer** examines flagged workflows, their source YAML, and recent logs. - It creates GitHub Issues with concrete inefficiencies and recommended fixes. - The auditing workflows also consume tokens, creating a feedback loop in which their own costs are monitored. ## Removing Unused MCP Tools - MCP tool names and JSON schemas are typically included in every stateless LLM request. - A GitHub MCP server with roughly 40 tools can add 10–15 KB of schema to every turn. - If a workflow uses only two tools, the other 38 create repeated overhead without adding value. - GitHub compares configured tools with actual tool calls and recommends removing unused registrations. - In smoke tests, pruning tools reduced each call’s context by 8–12 KB and saved several thousand tokens per run without changing behavior. ## Replacing MCP Calls with GitHub CLI - GitHub found larger savings by replacing MCP calls for predictable data retrieval—such as pull request diffs, file contents, and review comments—with `gh` commands. - MCP calls require an additional reasoning cycle: the model chooses a tool, constructs arguments, and processes the response. - Commands such as `gh pr diff` make deterministic API requests without involving the LLM in the retrieval step. Two migration patterns were used: - **Pre-agentic downloads** - Workflow setup steps run `gh` commands before the agent starts. - Results such as diffs and changed-file lists are saved to workspace files. - The agent reads the files directly, eliminating MCP round trips. - **In-agent CLI proxy substitution** - When data must be selected dynamically, the agent runs commands such as `gh pr view --json`. - A transparent proxy routes CLI requests to GitHub’s API without exposing credentials. - This preserves the zero-secrets security model while avoiding MCP overhead. ## Measuring Efficiency - Lower token counts do not necessarily mean better workflows; a workflow may simply be doing less work. - Model selection also affects cost. Claude Haiku and Sonnet may use similar numbers of tokens, but Haiku is substantially cheaper. - GitHub therefore uses an **Effective Tokens (ET)** metric that weights usage by token type and model cost: ```text ET = m × (1.0 × I + 0.1 × C + 4.0 × O) ``` - `m` represents the model multiplier: Haiku `0.25×`, Sonnet `1.0×`, and Opus `5.0×`. - `I` is newly processed input, `C` is cache-read tokens, and `O` is output tokens. - Output tokens receive greater weight because they are typically the most expensive component. GitHub’s experience suggests that agentic workflow authors should measure usage continuously, remove tools that workflows do not actually use, and move routine API retrieval outside the LLM reasoning loop wherever possible.

Read original

Model Context Protocol Large Language Models GitHub Actions Yaml+3

githubApr 15, 20262 min readCurated summary

Build a personal organization command center with GitHub Copilot CLI

Brittany Ellich built a personal organization command center to reduce the friction of switching between numerous apps. Using GitHub Copilot for planning and implementation, she created a working first version in one day. Her experience suggests that AI-assisted development makes it easier to turn everyday frustrations into practical, customized tools. ## The Problem: Digital Fragmentation - Brittany wanted to consolidate information scattered across roughly a dozen applications. - Her goal was a calm, visual workspace suited to her learning style and the way she organizes information. - The project became a personal command center for bringing disparate work and productivity tools together. ## Planning and Building with AI - Brittany uses a “plan-then-implement” workflow. - During planning, Copilot interviews her about requirements and behavior until they form a sufficiently detailed plan. - Copilot then implements the system based on that plan, reducing guesswork and making development smoother. - The first version was completed in a single day alongside her normal work. ## Her Development Workflow - She uses VS Code Agent Mode for synchronous work, generally running up to two non-competing agent workflows. - She uses Copilot Cloud Agent for asynchronous, well-scoped tasks such as bug fixes and technical-debt cleanup. - This division lets her focus personally on work requiring close oversight while agents handle lower-risk background tasks. ## Technology Stack - **Electron** for the cross-platform desktop application. - **React** for the interface, components, and state management. - **Vite** for development tooling and hot module replacement. - **Tailwind CSS** for styling. - **WorkIQ MCP and CLI** for accessing Microsoft 365 data, including calendar information. - The application also uses ElevenLabs for its voice assistant. Although Brittany had wanted to build an Electron app, she learned relatively little about Electron because Agent Mode handled most of the implementation. She later simplified the repository manually to make it suitable for public release, noting that agents tend to add code more readily than remove it. ## Getting Started The project is available as the open-source `command-center-lite` repository. Running it requires Node.js 18 or later, GitHub Copilot CLI for WorkIQ setup, a Microsoft 365 account for calendar synchronization, and an ElevenLabs account for voice features. The broader recommendation is to start building solutions for small, personal problems. AI tools can accelerate both learning and implementation, making experimentation with unfamiliar technologies far more accessible.

Read original

Model Context Protocol React Github Copilot Cli Node.Js+3

githubApr 14, 20263 min readCurated summary

Hack the AI agent: Build agentic AI security skills with the GitHub Secure Code Game

Agentic AI tools can automate powerful tasks, but their autonomy creates new security risks, including prompt injection, tool misuse, memory poisoning, and compromised multi-agent workflows. GitHub’s Season 4 Secure Code Game teaches developers to recognize these threats by attacking and hardening ProdBot, a deliberately vulnerable terminal-based AI assistant. Its five levels progressively add capabilities—and corresponding attack surfaces—mirroring how real-world AI systems evolve. ## The Secure Code Game’s Evolution - The free, open-source, in-editor course teaches security by having players exploit and fix intentionally vulnerable code. - Earlier seasons covered: - General secure coding across JavaScript, Python, Go, and GitHub Actions. - LLM security, including malicious prompts and defensive techniques. - More than 10,000 developers from industry, academia, and open source have participated. - Season 4 shifts focus from AI that generates content to AI that independently browses, uses tools, calls APIs, and acts for users. ## Why Agentic AI Security Is Urgent - Agentic systems are moving rapidly from research projects into production environments. - The OWASP Top 10 for Agentic Applications identifies threats such as: - Goal hijacking - Tool misuse - Identity abuse - Memory poisoning - A Dark Reading poll found that 48% of cybersecurity professionals expect agentic AI to be the leading attack vector by the end of 2026. - Cisco reported that although 83% of organizations planned to deploy agentic AI, only 29% felt prepared to secure it. - The article argues that learning to think like an attacker is essential for closing this readiness gap. ## ProdBot: A Deliberately Vulnerable AI Assistant - ProdBot is a terminal-based productivity and coding assistant inspired by tools such as OpenClaw and GitHub Copilot CLI. - It can: - Convert natural-language requests into bash commands. - Browse a simulated web. - Connect to MCP servers. - Run organization-approved skills. - Store persistent memory. - Coordinate multiple agents. - Players’ objective is to use natural language to make ProdBot reveal the contents of `password.txt`. - No prior AI or coding experience is required; all interaction takes place through the CLI. ## Five Progressive Attack Surfaces - **Level 1: Shell execution** - ProdBot runs generated bash commands in a sandbox. - The challenge is to determine whether the sandbox can be escaped. - **Level 2: Web browsing** - ProdBot reads simulated news, finance, sports, and shopping sites. - Untrusted web content introduces risks such as instruction hijacking and prompt injection. - **Level 3: MCP integrations** - ProdBot gains access to external tool providers for stock quotes, browsing, and cloud backup. - Additional tools increase both functionality and opportunities for abuse. - **Level 4: Skills and memory** - Organization-approved plugins and persistent memory create layered trust relationships. - The level tests whether trusted skills and stored information are actually safe. - **Level 5: Multi-agent orchestration** - ProdBot combines six specialized agents, three MCP servers, three skills, and a simulated open-source project. - Claims that agents are sandboxed and data is pre-verified become assumptions to test rather than guarantees. ## Real-World Relevance - The game’s vulnerabilities reflect active security concerns in deployed autonomous AI systems rather than purely theoretical exercises. - The article cites CVE-2026-25253, known as “ClawBleed,” an OpenClaw vulnerability rated CVSS 8.8. - The flaw allowed attackers to steal authentication tokens through a malicious link and gain full control of an OpenClaw instance. - Season 4’s broader goal is to develop instincts for identifying similar weaknesses during architecture reviews, tool-integration audits, and production deployments. Developers working with AI agents should treat every new capability—shell access, browsing, plugins, memory, or collaboration—as a potential attack surface. Practicing these failure modes in a controlled environment like the Secure Code Game can help teams design safer agentic systems before deploying them.

Read original

Model Context Protocol AI Agents Ai Security Prompt Injection+2

githubApr 10, 20263 min readCurated summary

GitHub Copilot CLI for Beginners: Getting started with GitHub Copilot CLI

GitHub Copilot CLI brings Copilot’s agentic coding capabilities directly into the terminal, allowing developers to inspect projects, generate code, run tests, and correct errors without switching tools. The post introduces the tool, explains installation and authentication, and demonstrates how to use it for project overviews, coding tasks, and delegated work. Its central message is that Copilot CLI can preserve development flow while supporting increasingly autonomous coding workflows. ## What GitHub Copilot CLI Does - Runs Copilot from a command-line interface with context from the current repository. - Can autonomously: - Build or modify code - Run tests - Detect and correct errors - Explore project files and documentation - Lets developers review results and request follow-up changes directly in the terminal. - Can delegate well-defined tasks to the Copilot cloud agent. ## Installing Copilot CLI - The primary cross-platform installation method, assuming Node.js is available, is: ```bash npm install -g @github/copilot ``` - Users can also install it through package managers such as Homebrew or WinGet. ## First-Time Setup - Launch the tool by entering `Copilot` in the terminal. - Authenticate with GitHub using: ```plaintext /login ``` - Authentication connects the CLI to the user’s Copilot account and the read-only GitHub MCP server. - Copilot must be granted permission to access the current folder so it can inspect or modify files. - Folder permissions can apply only to the current session or be saved for future sessions. ## Common Development Tasks - **Understand an existing project** - Prompt Copilot with: ```plaintext Give me an overview of this project ``` - It examines important files and summarizes the project structure and purpose. - **Generate new code** - For example: ```plaintext Let’s add a new endpoint to return all categories ``` - Copilot reviews existing conventions, documentation, and examples before proposing or creating files. - It requests permission before making changes. - **Delegate work to the cloud agent** - A task can be sent using: ```plaintext /delegate Let’s deal with issue #14 to add the rest of the CRUD endpoints to games ``` - The cloud agent retains the current context, creates a branch, opens a draft pull request, and performs the work in the background for later review. ## What Comes Next The broader beginner series will cover interactive mode, non-interactive mode using the `-p` flag, slash commands, and MCP server integration. These features expand Copilot CLI from an interactive coding assistant into a flexible terminal-based automation tool. Copilot CLI is recommended for developers who want AI assistance without leaving the shell: install it with npm, authenticate, grant project permissions, and begin with exploratory prompts before assigning code changes or delegated tasks.

Read original

Model Context Protocol Github Copilot Cli Node.Js Npm+3

githubMar 31, 20263 min readCurated summary

Agent-driven development in Copilot Applied Science

The post describes how Tyler McGoffin used GitHub Copilot to automate the intellectual work of analyzing coding-agent evaluation trajectories. This led to `eval-agents`, a tool designed to let researchers create, share, and run specialized agents. By making coding agents the primary contributors, the team rapidly added 11 agents, four skills, and workflow support while learning new approaches to prompting, architecture, and collaboration. ## The Motivation: Automating Evaluation Analysis - McGoffin analyzes coding-agent performance using benchmarks such as TerminalBench2 and SWEBench-Pro. - Each benchmark task produces a trajectory: a large JSON record of the agent’s thoughts and actions. - Reviewing hundreds or thousands of trajectories can involve hundreds of thousands of lines of data. - Copilot initially helped identify patterns, reducing the amount of material requiring manual inspection from hundreds of thousands of lines to a few hundred. - The repetitive nature of this process inspired `eval-agents`, which automates parts of the analysis itself. ## Project Goals The project was designed around three objectives: - Make agents easy for others to share and use. - Make authoring new agents straightforward. - Make coding agents the primary mechanism for contributing to the project. The third goal had the greatest architectural impact. Using Copilot to build the tool also made the repository easier for teammates to understand, extend, and collaborate on. ## An Agent-First Development Setup McGoffin’s development environment consisted of: - Copilot CLI as the coding agent. - Claude Opus 4.6 as the model. - VS Code as the IDE. - The Copilot SDK for creating agents, registering tools and skills, and accessing existing MCP servers. This setup allowed the project to reuse Copilot’s existing agent infrastructure instead of implementing those capabilities from scratch. ## Prompting Strategies - Agents perform best when treated like capable engineers rather than simple code generators. - Effective prompts are conversational, detailed, and explicit about assumptions. - Planning mode should be used before implementation mode, especially for complex tasks. - McGoffin used stream-of-consciousness descriptions to explain problems and collaborate with Copilot on possible solutions. - For example, a discussion about preventing agents from weakening regression tests led to protected test areas and human-controlled contract-test-like guardrails. - The broader lesson is that agents benefit from many of the same practices as human engineers: context, dialogue, planning, and clear constraints. ## Architectural Strategies An agent-first codebase makes maintainability work especially valuable: - Refactoring names and file structures improves the repository’s understandability. - Documentation gives agents the context needed to implement features consistently. - Additional tests expose and prevent recurring mistakes. - Removing dead code helps keep agents from copying outdated or irrelevant patterns. - Work that was traditionally postponed—cleanup, documentation, and test improvements—becomes foundational when agents are responsible for much of the implementation. ## Rapid Team Collaboration Applying these principles enabled substantial development in a short period: - Five people contributed to the project for the first time. - The team created 11 agents and four skills. - They introduced eval-agent workflows for structured streams of scientific reasoning. - In under three days, the changes amounted to approximately 28,858 added and 2,884 removed lines across 345 files. ## Practical Recommendation Teams adopting agent-driven development should invest first in clear architecture, documentation, tests, and conversational planning practices. Agents become substantially more effective when the repository provides strong context and guardrails, allowing developers to focus less on repetitive implementation and more on directing, reviewing, and improving the overall system.

Read original

Model Context Protocol AI Agents Github Copilot Copilot Cli+2

githubMar 10, 20262 min readCurated summary

The era of “AI as text” is over. Execution is the new interface.

The post argues that AI is moving beyond text-based question-and-answer interactions toward embedded execution. The GitHub Copilot SDK lets applications use Copilot’s planning, tool use, file modification, command execution, and error recovery capabilities directly. This enables teams to build adaptable AI workflows without creating their own orchestration infrastructure. ## Delegating Multi-Step Work to Agents - Applications can express intent and constraints instead of hard-coding every workflow step. - For a task such as “Prepare this repository for release,” an agent can: - Explore the repository - Plan the necessary work - Modify files - Run commands - Recover and adapt when failures occur - This approach is more flexible than scripts, which become brittle when workflows depend on changing context or unexpected errors. - Teams can use agentic execution while maintaining defined boundaries and observability. ## Using Structured Runtime Context - Relying on prompts to contain system logic makes workflows difficult to test, maintain, and evolve. - The Copilot SDK supports structured, composable context through: - Domain-specific tools and agent skills - Model Context Protocol (MCP) - Runtime retrieval of relevant data - Agents can directly access systems such as: - Service ownership records - Historical decisions - Dependency graphs - Internal APIs - Permissioned tools and real-time data provide more reliable grounding than embedding organizational knowledge in prompts. ## Embedding Agents Beyond the IDE - Agentic capabilities can be integrated into: - Desktop applications - Internal operational tools - Background services - SaaS products - Event-driven systems - Applications can invoke Copilot in response to events such as file changes, deployments, or user actions. - Execution happens within the product itself rather than in a separate IDE or terminal interface. - This turns AI from an auxiliary developer tool into application infrastructure available wherever the software operates. ## Execution as a New Interface - Agentic workflows are programmable planning-and-execution loops that: - Integrate with real systems - Operate under constraints - Adapt during runtime - Use tools to complete tasks - The Copilot SDK provides this execution layer so teams can focus on defining outcomes instead of rebuilding orchestration systems. The practical recommendation is to treat AI as an executable application capability rather than merely a text interface. Teams can start by identifying multi-step workflows or event-driven tasks where structured tools, runtime context, and adaptive execution would provide more value than fixed scripts.

Read original

Model Context Protocol AI Agents Github Copilot Agentic Workflows+2

githubMar 9, 20263 min readCurated summary

Under the hood: Security architecture of GitHub Agentic Workflows

GitHub Agentic Workflows are designed to bring autonomous agents into CI/CD without giving them unrestricted access to repositories, secrets, or the internet. Because agents can be prompt-injected and behave unpredictably, GitHub treats them as untrusted components and compiles workflows into constrained GitHub Actions. The architecture relies on layered isolation, controlled communication, staged writes, and comprehensive auditing. ## Threat Model - Agents reason over repository state and act autonomously, so they cannot be trusted by default. - GitHub Actions normally place components in one permissive trust domain with broad access to: - Repository contents - Authentication secrets - MCP servers - Arbitrary network destinations - A malicious webpage, issue, or repository file could prompt an agent to: - Read credentials from files, environment variables, logs, or `/proc` - Upload secrets externally - Embed secrets in issues, pull requests, or comments - Make unwanted repository changes - Strict mode follows four principles: - Defense in depth - Never trust agents with secrets - Stage and vet writes - Log everything ## Layered Security Architecture GitHub Agentic Workflows use three complementary layers: - **Substrate layer** - Runs on a GitHub Actions runner VM. - Uses trusted containers, Docker isolation, network controls, and kernel-enforced boundaries. - Separates components and mediates privileged operations and system calls. - Is intended to contain damage even if an untrusted component is compromised. - **Configuration layer** - Defines which components run and how they connect. - Controls communication channels, privileges, firewall policies, Docker images, and MCP configuration. - Determines which tokens are loaded into which containers. - Converts declarative workflow configuration into a secure runtime structure. - **Planning layer** - Controls which components are active and how data moves between them over time. - Creates staged workflows with explicit data exchanges. - Uses the Safe Outputs subsystem to govern potentially dangerous operations. ## Keeping Secrets Away from Agents - In ordinary GitHub Actions, secrets may be visible through environment variables and configuration files across the shared runner trust domain. - This creates a major prompt-injection risk: an agent with shell access could discover credentials and exfiltrate them. - Agentic Workflows instead place the agent in a dedicated container with: - Firewalled internet access - MCP access through a trusted gateway - LLM communication through an API proxy - A private network connects the agent only to approved services. - The trusted MCP gateway launches MCP servers and exclusively handles MCP authentication material. - LLM authentication tokens are kept in the isolated API proxy rather than exposed directly inside the agent container. ## Controlled Execution and Writes - Open-ended workflow authoring is separated from governed execution. - Workflows are compiled into GitHub Actions with explicit constraints covering: - Permissions - Outputs - Network access - Auditability - The planning and Safe Outputs systems are intended to mediate GitHub write operations and apply controls such as call filtering, volume limits, secret removal, and moderation. GitHub’s approach is to treat agents as untrusted CI/CD components rather than granting them normal workflow privileges. Organizations adopting agentic automation should isolate agents, broker access to tools and credentials, restrict network connectivity, stage all writes for review, and maintain detailed logs.

Read original

Model Context Protocol CI/CD Docker & Containers GitHub Actions+3

githubFeb 24, 20262 min readCurated summary

Multi-agent workflows often fail. Here’s how to engineer ones that don’t.

Multi-agent workflows often fail because agents make implicit assumptions about state, ordering, and intended actions. The post argues that these systems should be engineered like distributed software rather than treated as chat interfaces. Typed schemas, explicit action definitions, and MCP-enforced interfaces make agent behavior more predictable and failures easier to contain. ## Typed Schemas Prevent Data Drift - Natural-language exchanges and inconsistent JSON lead to changing field names, mismatched types, and ambiguous payloads. - Typed interfaces define machine-checkable contracts, such as a `UserProfile` with fixed fields and allowed plan values. - Schema violations can fail fast, triggering retries, repairs, or escalation before invalid state spreads. - Debugging becomes contract-based instead of dependent on inspecting logs and guessing. ## Action Schemas Clarify Intent - Agents cannot reliably infer what “take action” means; they may assign, close, escalate, or do nothing. - Action schemas restrict outcomes to explicit, valid choices such as: - Requesting more information - Assigning an issue - Closing an issue as a duplicate - Taking no action - A discriminated union or similar structure ensures every agent returns one recognized action. - Invalid or ambiguous actions can be rejected, retried, or escalated. ## MCP Enforces Agent Interfaces - Schemas and action definitions are only conventions unless consistently enforced. - Model Context Protocol (MCP) provides explicit input and output schemas for tools and resources. - Calls are validated before execution, preventing agents from inventing fields, omitting required inputs, or drifting between interfaces. - MCP therefore acts as the enforcement layer for both data structure and intended behavior. Reliable multi-agent systems require explicit contracts at every boundary. Engineers should treat agents like code components: define their data and actions precisely, enforce interfaces with mechanisms such as MCP, and prevent invalid state from propagating.

Read original

Model Context Protocol AI Agents Github Copilot Distributed Systems+2