AI Agents | Techlist.io

woowahanDec 4, 2025Original article

Test Automation with AI: Plugin Development Story (opens in new tab)

This blog post explores how a development team at Woowahan Tech successfully automated the creation of 100 unit tests in just 30 minutes by combining a custom IntelliJ plugin with Amazon Q. The author argues that while full AI automation often fails in complex multi-module environments, a hybrid approach using "compile-guaranteed templates" ensures high success rates and maintains operational stability. This strategy allows developers to bypass repetitive setup tasks while leveraging AI for logic implementation within a strictly defined, valid structure. ### Evaluating AI Assistants for Testing * The team compared various AI tools including GitHub Copilot, Cursor, and Amazon Q to determine which best fit their existing IntelliJ-based workflow. * Amazon Q was selected for its superior understanding of the entire project context and its ability to integrate seamlessly as a plugin without requiring a switch to a new IDE. * Initial manual use of AI assistants highlighted repetitive patterns: developers had to constantly specify team conventions (Kotest FunSpec, MockK) and manually fix build errors in 15% of the generated code. * On average, it took 10 minutes per class to generate and refine tests manually, prompting the team to seek a more automated solution via a custom plugin. ### The Pitfalls of Full Automation * The first version of the custom plugin attempted to generate complete test files by gathering class metadata through PSI (Program Structure Interface) and sending it to the Gemini API. * Pilot tests revealed a 90% compilation failure rate, as the AI frequently generated incorrect imports, hallucinated non-existent fields, or used mismatched data types. * A critical issue was the "loss of existing tests," where the AI-generated output would completely overwrite previous work rather than appending to it. * In complex multi-module projects, the AI struggled to identify the correct classes when multiple modules contained identical class names, leading to significant manual correction time. ### Shifting to Compile-Guaranteed Templates * To overcome the limitations of full automation, the team pivoted to a "template first" approach where the plugin generates a valid, compilable shell for the test. * The plugin handles the complex infrastructure of the test file, including correct imports, MockK setups, and empty test stubs for every method in the target class. * This approach reduces the AI's "hallucination surface" by providing it with a predefined structure, allowing tools like Amazon Q to focus solely on filling in the implementation details. * By automating the 1-minute setup and letting the AI handle the 2-minute implementation phase, the team achieved a 97% success rate across 100 test cases. ### Practical Conclusion For teams looking to improve test coverage in large-scale repositories, the most effective strategy is to use IDE plugins to automate context gathering and boilerplate generation. By providing the AI with a structurally sound template, developers can eliminate compilation errors and significantly reduce the time spent on manual refinement, ensuring that even complex edge cases are covered with minimal effort.

AI Agents AI Kotlin Test Automation+5

microsoftDec 3, 20253 min readCurated summary

The Interaction Changes Everything: Treating AI Agents as Collaborators, Not Automation

The article argues that effective AI agents should be treated as engineering collaborators, not automation scripts. Microsoft applied this approach to migrate hundreds of repositories from Entra SDK v1 to v2, reducing work from 4–6 weeks per repository to under two hours with 80–90% accuracy. The key improvement came from giving the agent a role, mission, priorities, and permission to exercise judgment. ## The Entra SDK Migration Challenge - The migration involved hundreds of repositories and sensitive authentication security boundaries. - Traditional migrations required extensive human review and took 4–6 weeks per repository. - The AI agent completed comparable work in under two hours while achieving 80–90% accuracy. - The goal was not merely speed, but reliable handling of custom configurations, edge cases, and security concerns. ## Problems with Automation Thinking - Initial attempts treated the agent like a script executor: - Detailed transformation instructions were provided. - Every anticipated edge case was documented. - The agent was expected to follow a checklist. - This approach repeatedly failed because complex migrations require: - Context-dependent decisions - Handling of undocumented patterns - Security-boundary evaluation - Trade-offs between correctness, speed, and preservation of custom logic - The central lesson is that judgment cannot be fully automated, but it can be supported through collaboration with an intelligent agent. ## Identity Instead of Instructions - The team reframed the agent as a member of the migration team rather than a tool. - The prompt described the agent as a “co-creative engineer” expected to: - Use judgment - Stay curious - Act carefully - Ask for help when uncertain - This change improved accuracy and edge-case handling. - The agent was more likely to surface uncertainty instead of guessing or failing silently. ## The Co-Creative Partnership Framework ### Identity and Mission - Establish the agent’s team, mission, and the importance of the work. - Explicitly state that the agent is not a script executor. - Explain why the task matters so the agent can prioritize appropriately. - Encourage judgment, curiosity, and care. ### Purpose and Intent - Describe the guide as supporting both human and AI team members. - Make priorities explicit, such as security over speed or correctness over completion. - Allow autonomy when repository contexts differ. - Frame uncertainty as a reason to collaborate rather than as failure. ### Prioritized Goals - List primary, secondary, quality, and human-in-the-loop objectives in order. - Explicit priorities help the agent resolve conflicts. - Including quality and collaboration prevents optimizing for speed alone. ### Step-by-Step Guidance with Judgment - Provide concrete actions, conditional logic, edge-case handling, and before-and-after examples. - Specify what must remain unchanged, including custom logic. - Define situations requiring escalation, such as unusual patterns, ambiguity, or possible security violations. - The framework combines procedural guidance with room for context-sensitive decisions. ## Practical Recommendation For complex migrations, security reviews, or architectural work, write prompts that define a collaborative role and decision-making framework—not just a list of commands. Give the agent context, priorities, preservation rules, examples, and clear escalation points so it can act autonomously while knowing when human judgment is required.

Read original

AI Agents Github Copilot Prompt Engineering Software Development Lifecycle+3

awsDec 2, 2025Original article

Amazon Bedrock AgentCore adds quality evaluations and policy controls for deploying trusted AI agents (opens in new tab)

AWS has introduced several new capabilities to Amazon Bedrock AgentCore designed to remove the trust and quality barriers that often prevent AI agents from moving into production environments. These updates, which include granular policy controls and sophisticated evaluation tools, allow developers to implement strict operational boundaries and monitor real-world performance at scale. By balancing agent autonomy with centralized verification, AgentCore provides a secure framework for deploying highly capable agents across enterprise workflows. **Governance through Policy in AgentCore** * This feature establishes clear boundaries for agent actions by intercepting tool calls via the AgentCore Gateway before they are executed. * By operating outside of the agent’s internal reasoning loop, the policy layer acts as an independent verification system that treats the agent as an autonomous actor requiring permission. * Developers can define fine-grained permissions to ensure agents do not access sensitive data inappropriately or take unauthorized actions within external systems. **Quality Monitoring with AgentCore Evaluations** * The new evaluation framework allows teams to monitor the quality of AI agents based on actual behavior rather than theoretical simulations. * Built-in evaluators provide standardized metrics for critical dimensions such as helpfulness and correctness. * Organizations can also implement custom evaluators to ensure agents meet specific business-logic requirements and industry-specific compliance standards. **Enhanced Memory and Communication Features** * New episodic functionality in AgentCore Memory introduces a long-term strategy that allows agents to learn from past experiences and apply successful solutions to similar future tasks. * Bidirectional streaming in the AgentCore Runtime supports the deployment of advanced voice agents capable of handling natural, simultaneous conversation flows. * These enhancements focus on improving consistency and user experience, enabling agents to handle complex, multi-turn interactions with higher reliability. **Real-World Application and Performance** * The AgentCore SDK has seen rapid adoption with over 2 million downloads, supporting diverse use cases from content generation at the PGA TOUR to financial data analysis at Workday. * Case studies highlight significant operational gains, such as a 1,000 percent increase in content writing speed and a 50 percent reduction in problem resolution time through improved observability. * The platform emphasizes 100 percent traceability of agent decisions, which is critical for organizations transitioning from reactive to proactive AI-driven operations. To successfully scale AI agents, organizations should transition from simple prompt engineering to a robust agentic architecture. Leveraging these new policy and evaluation tools will allow development teams to maintain the necessary control and visibility required for customer-facing and mission-critical deployments.

AI Agents AI Generative AI Observability+5

slackDec 1, 20253 min readCurated summary

Streamlining Security Investigations with Agents

Slack’s Security Engineering team uses a multi-agent AI system to investigate security alerts across billions of daily events. After finding that a single prompt produced inconsistent results, the team replaced it with a controlled workflow of specialized model invocations, structured outputs, and application-level orchestration. The resulting system improves consistency, enables evidence review, and allocates more capable models only where they add the most value. ## From Prompt Prototype to Controlled Workflow - The initial prototype consisted of a roughly 300-word prompt defining: - The analyst’s role - Available data sources - Investigation methodology - Report formatting - Response classifications - A stdio-based MCP server safely exposed selected security data sources to the model. - A coding-agent CLI served as the prototype execution environment. - Results varied significantly: - Sometimes the model cross-referenced evidence effectively. - Other times it reached convenient or unsupported conclusions too quickly. - Prompt refinements helped somewhat, but prompts were ultimately too limited for fine-grained process control. ## Structured Investigation Tasks - Slack decomposed the investigation into a sequence of model invocations. - Each invocation performs one well-defined task and returns a structured output. - The application chains these tasks together and passes only the necessary context between stages. - Structured outputs use JSON schemas to constrain model responses. - They improve predictability, but can still fail when schemas are too complex and remain vulnerable to hallucination or attempts to circumvent constraints. - Guidance such as “question your evidence” became an explicit workflow step rather than merely an instruction in a prompt. ## Persona-Based Agent Architecture - Slack drew inspiration from research on meta-prompting, multi-persona collaboration, and security tabletop exercises. - Instead of representing multiple personas within one model call, Slack implemented each persona as an independent model invocation. - Every agent/task pair has: - A defined responsibility - A carefully designed output structure - Specific prompts and instructions - Potentially different models and tools - The application orchestrates the agents and controls how knowledge moves through the investigation. ## The Investigation Loop - **Director agent** - Guides the investigation from beginning to end. - Forms questions for domain experts. - Uses a journaling tool to plan and organize progress. - Decides how to continue based on reviewed findings and timelines. - **Expert agents** - Investigate questions using specialized knowledge and data sources. - Slack currently uses four domains: - **Access:** Authentication, authorization, and perimeter services - **Cloud:** Infrastructure, compute, orchestration, and networking - **Code:** Source code and configuration management - **Threat:** Threat intelligence and analysis - **Critic agent** - Acts as a meta-expert reviewing domain findings. - Applies a defined rubric to assess quality. - Adds analysis and credibility scores to individual findings. - Helps identify reliable evidence and reduce hallucinations. - Returns its conclusions to the Director, closing the investigation loop. - The Critic’s mildly adversarial role provides an independent challenge to expert conclusions. ## The Knowledge Pyramid - Different stages use different model capabilities and costs. - Domain experts operate at the base: - They query complex data sources. - They may make many tool calls. - Processing their results can consume substantial tokens. - The Critic reviews the larger set of expert findings and identifies the most valuable or credible evidence. - Higher-level reasoning can therefore use a smaller, more focused context. - Because each stage is independent, Slack can select different model versions, prompts, tools, and output formats for different tasks. Slack’s main recommendation is to treat complex agent behavior as an explicitly orchestrated workflow rather than relying on one increasingly elaborate prompt. Specialized agents, structured outputs, independent critique, and tiered model usage provide more predictable investigations while preserving the flexibility of AI-assisted analysis.

Read original

AI Agents Large Language Models Model Context Protocol Prompt Engineering+3

kakaoNov 23, 2025Original article

[AI_TOP_10 (opens in new tab)

The AI TOP 100 contest was designed to shift the focus from evaluating AI model performance to measuring human proficiency in solving real-world problems through AI collaboration. By prioritizing the "problem-solving process" over mere final output, the organizers sought to identify individuals who can define clear goals and navigate the technical limitations of current AI tools. The conclusion of this initiative suggests that true AI literacy is defined by the ability to maintain a "human-in-the-loop" workflow where human intuition guides AI execution and verification. ### Core Philosophy of Human-AI Collaboration * **Human-in-the-Loop:** The contest emphasizes a cycle of human analysis, AI problem-solving, and human verification. This ensures that the human remains the "pilot" who directs the AI engine and takes responsibility for the quality of the result. * **Strategic Intervention:** Participants were encouraged to provide AI with structural context it might struggle to perceive (like complex table relationships) and to perform data pre-processing to improve AI accuracy. * **Task Delegation:** For complex iterative tasks, such as generating images for a montage, solvers were expected to build automated pipelines using AI agents to handle repetitive feedback loops while focusing human effort on higher-level strategy. ### Designing Against "One-Shot" Solutions * **Low Barrier, High Ceiling:** Problems were designed to be intuitive enough for anyone to understand but complex enough to prevent "one-shot" solutions (the "click-and-solve" trap). * **Targeting Technical Weaknesses:** Organizers intentionally embedded technical hurdles that current LLMs struggle with, forcing participants to demonstrate how they bridge the gap between AI limitations and a correct answer. * **The Difficulty Ladder:** To account for varying domain expertise (e.g., OCR experience), problems utilized a multi-part structure. This included "Easy" starting questions to build momentum and "Medium" hint questions that guided participants toward solving the more difficult "Killer" components. ### The 4-Pattern Problem Framework * **P1 - Insight (Analysis & Definition):** Identifying meaningful opportunities or problems within complex, unstructured data. * **P2 - Action (Implementation & Automation):** Developing functional code or workflows to execute a defined solution. * **P3 - Persuasion (Strategy & Creativity):** Generating logical and creative content to communicate technical solutions to non-technical stakeholders. * **P4 - Decision (Optimization):** Making optimal choices and simulations to maximize goals under specific constraints. ### Quality Assurance and Score Calibration * **4-Stage Pipeline:** Problems moved from Ideation to Drafting (testing for one-shot immunity), then to Candidate (analyzing abuse vulnerabilities), and finally to a Final selection based on difficulty balance. * **Cross-Model Validation:** Internal and alpha testers solved problems using various models including Claude, GPT, and Gemini to ensure that no single tool could bypass the intended human-led process. * **Effort-Based Scoring:** Instead of uniform points, scores were calibrated based on the "effort cost" and human competency required to solve them. This resulted in varying total points per problem to better reflect the true difficulty of the task. In the era of rapidly evolving AI, the ability to "use" a tool is becoming less valuable than the ability to "collaborate" with it. This shift requires a move toward building automated pipelines and utilizing a "difficulty ladder" approach to tackle complex, multi-stage problems that AI cannot yet solve in a single iteration.

AI Agents AI Large Language Models Automation+4

dropboxNov 17, 20253 min readCurated summary

How Dash uses context engineering for smarter AI

Dash evolved from a traditional RAG search system into an agentic AI that can interpret information, plan tasks, and act on users’ behalf. Dropbox’s experience shows that better agent performance comes not from adding more tools and data, but from carefully engineering context: limiting choices, filtering for relevance, and delegating complex work to specialized agents. The central conclusion is that precise, timely context improves reasoning speed, accuracy, and efficiency. ## From Search to Agentic AI - Dash initially combined semantic and keyword search to retrieve documents and generate concise answers. - Users began asking it to interpret, summarize, and act on retrieved information. - This required Dash to plan and execute multi-step tasks rather than simply search and summarize. - The resulting challenge was determining which information and tools the model actually needed at each stage. ## The Cost of Too Many Tools - Every tool adds descriptions and parameters to the model’s context window. - More tools expand the model’s decision space, potentially causing slower or less reliable choices. - Tool definitions also consume tokens, increasing cost and reducing room for reasoning. - Longer-running tasks suffered from “context rot,” where accumulated tool-call information degraded accuracy. - Model Context Protocol (MCP) standardizes tool descriptions, but does not eliminate the problem of excessive context. ## Limiting Tool Definitions - Dash found that exposing retrieval tools from many services—such as Confluence, Google Docs, and Jira—created confusion. - Instead of requiring the model to choose among numerous APIs, Dash consolidated retrieval into one purpose-built tool backed by its universal search index. - A single retrieval interface: - Simplifies planning - Reduces tool-selection errors - Keeps the context window focused - Provides consistent access across connected services - The same principle shaped Dash’s MCP server, which exposes retrieval through one lean tool to applications such as Claude, Cursor, and Goose. ## Filtering Context for Relevance - Retrieved information is not automatically useful for the task at hand. - Dash combines data from multiple sources in a unified index and uses a knowledge graph to connect people, activity, and content. - These relationships help rank results according to the query and the user’s context. - By filtering results before presenting them to the model, Dash ensures that each piece of supplied context is relevant. - Precomputing the index and graph allows runtime retrieval to remain fast and focused. ## Using Specialized Agents for Complex Tasks - Some tools require substantial instructions and examples to use correctly. - Dash Search became complex because query construction involves: - Understanding user intent - Mapping intent to index fields - Rewriting queries for semantic matching - Handling typos, synonyms, and implicit context - Adding these instructions directly to the main planning agent consumed context that could otherwise support broader reasoning. - Dash therefore moved search into a specialized agent: - The main agent decides when searching is necessary. - The search agent independently constructs the query using its dedicated prompt. - This division lets the main agent focus on the overall task while the specialist handles search details. Dash’s approach recommends treating context as a limited engineering resource. Use a small number of well-designed tools, pre-filter information for relevance, and delegate technically demanding subtasks to specialized agents rather than overwhelming one general-purpose model.

Read original

AI Agents Model Context Protocol RAG Context Engineering+3

googleSep 24, 2025Original article

Towards better health conversations: Research insights on a “wayfinding” AI agent based on Gemini (opens in new tab)

Google Research has developed "Wayfinding AI," a research prototype based on Gemini designed to transform health information seeking from a passive query-response model into a proactive, context-seeking dialogue. By prioritizing clarifying questions and iterative guidance, the agent addresses the common struggle users face when attempting to articulate complex or ambiguous medical concerns. User studies indicate that this proactive approach results in health information that participants find significantly more helpful, relevant, and tailored to their specific needs than traditional AI responses. ### Challenges in Digital Health Navigation * Formative research involving 33 participants highlighted that users often struggle to articulate health concerns because they lack the clinical background to know which details are medically relevant. * The study found that users typically "throw words" at a search engine and sift through generic, impersonal results that do not account for their unique context. * Initial UX testing revealed a strong user preference for a "deferred-answer" approach, where the AI mimics a medical professional by asking clarifying questions before jumping to a conclusion. ### Core Design Principles of Wayfinding AI * **Proactive Conversational Guidance:** At every turn, the agent asks up to three targeted questions to reduce ambiguity and help users systematically share their "health story." * **Best-Effort Answers:** To ensure immediate utility, the AI provides the best possible information based on the data available at that moment, while noting that the answer will improve as the user provides more context. * **Transparent Reasoning:** The system explicitly explains how the user’s most recent answers have helped refine the previous response, making the AI’s internal logic understandable. ### Split-Stream User Interface * To prevent clarifying questions from being buried in long paragraphs, the prototype uses a two-column layout. * The left column is dedicated to the interactive chat and specific follow-up questions to keep the user focused on the dialogue. * The right column displays the "best information so far" and detailed explanations, allowing users to dive into the technical content only when they feel enough context has been established. ### Comparative Evaluation and Performance * A randomized study with 130 participants compared the Wayfinding AI against a baseline Gemini 2.5 Flash model. * Participants interacted with both models for at least three minutes regarding a personal health question and rated them across six dimensions: helpfulness, question relevance, tailoring, goal understanding, ease of use, and efficiency. * The proactive agent outperformed the baseline significantly, with participants reporting that the context-seeking behavior felt more professional and increased their confidence in the AI's suggestions. The research suggests that for sensitive and complex topics like health, AI should move beyond being a passive knowledge base. By adopting a "wayfinding" strategy that guides users through their own information needs, AI agents can provide more personalized and empowering experiences that better mirror expert human consultation.

AI Agents AI Large Language Models Gemini+4

figmaSep 23, 20252 min readCurated summary

Why You Should Care About Design Context | Figma Blog

Design files are becoming an important input for agentic coding tools, so their organization and clarity directly affect the quality of generated code. Figma argues that designers should treat handoff as carefully as the design itself, providing developers and AI agents with explicit links between visual decisions and implementation details. The central recommendation is to build strong design-system foundations and structure files so they are easy for both humans and tools to interpret. ## Design systems as implementation foundations - Components and shared variables for color, spacing, sizing, and typography promote consistency and reuse. - These foundations become more valuable when connected directly to the codebase: - **Code Connect** links Figma components to their coded equivalents and displays accurate snippets in Dev Mode. - **Variable code syntax** defines how design variables should be represented across web and native platforms. - With the Dev Mode MCP server, AI agents can retrieve this design-system context directly instead of searching through an entire codebase. - A component such as a card can therefore be generated with the correct variants, properties, spacing, colors, typography, and states. - Embedding this information in shared Figma libraries allows the benefits to scale across teams and projects. ## Structuring files for developer consumption - Clear page hierarchies, frame names, layer names, and layouts make files easier to work with during design and easier for developers and MCP-powered tools to understand. - Create frames for important responsive breakpoints and major visual changes, such as a navigation bar becoming a menu icon. - Use auto layout to express how designs should respond to viewport widths, breakpoints, and changing text. - Intentional structure turns a design file into useful implementation context rather than merely a visual reference. Designers can improve development speed and AI-generated code by treating Figma files as structured technical documentation. Establish reusable, code-connected foundations first, then organize layouts and responsive states explicitly so developers and coding agents can reliably translate designs into production.

Read original

AI Agents Figma Design Systems Model Context Protocol+3

figmaSep 22, 20253 min readCurated summary

Design Context, Everywhere You Build | Figma Blog

Figma’s latest MCP server and Code Connect updates aim to make design-system and codebase context available wherever teams build. Remote MCP access connects Figma to IDEs, AI agents, and browser-based tools, while Figma Make integration exposes underlying prototype code. New in-app component mapping further links designs to production components, helping AI generate more consistent, production-aware code. ## Remote Access to the Figma MCP Server - The MCP server previously ran locally through Figma Dev Mode; it now supports remote access. - Developers can connect Figma to IDEs, AI coding agents, and browser-based models without installing the Figma desktop app. - Figma context can include responsive layouts, interaction details, design-system rules, and visual-program structure—not just static images. - Integrations are available through Figma’s partner catalog, with support involving tools such as Android Studio, Replit, and Warp. - Figma plans to expand the server’s functionality and partner ecosystem during and after its beta. - Companies including Affirm and Coinbase report significantly faster development, with Affirm rebuilding major product flows in under two days. ## Connecting Figma Make to Codebases - The MCP server can now expose the underlying code in Figma Make files rather than only rendered prototypes. - It indexes Make-file code so developers and AI agents can request specific files, reuse code, or reference its logic and design patterns. - The feature is available through partners including Anthropic, Cursor, Windsurf, and VS Code. - Figma is also working toward allowing partner MCP servers to connect into Figma Make. ## Improved Component Mapping with Code Connect - Code Connect links Figma components to their corresponding production components and usage guidelines. - A new in-app mapping workflow lets users browse Figma components, select the correct code and file, and identify mapped or missing relationships. - This complements the Code Connect CLI, which provides production-ready snippets and prop or variant mappings across frameworks. - When combined with MCP, Code Connect gives AI agents both design context and production awareness, including component locations and usage rules. - Figma’s internal evaluations and early customer testing showed more consistent code, faster file navigation, and better token efficiency. ## Integration Reliability - Figma is also introducing integration reviews and rate limits as it expands connections with external tools. - These measures are intended to make the growing extensibility ecosystem more reliable. Figma’s overall direction is to make design context portable and actionable across the development workflow. Teams can get the most value by combining remote MCP access for broad design understanding with Code Connect for accurate, production-ready component usage.

Read original

AI Agents Design Systems Model Context Protocol Code Generation+3

googleSep 18, 2025Original article

Deep researcher with test-time diffusion (opens in new tab)

Google Cloud researchers have introduced Test-Time Diffusion Deep Researcher (TTD-DR), a framework that treats long-form research report writing as an iterative diffusion process. By mimicking human research patterns, the system treats initial drafts as "noisy" versions that are gradually polished through retrieval-augmented denoising and self-evolutionary algorithms. This approach achieves state-of-the-art results in generating comprehensive academic-style reports and solving complex multi-hop reasoning tasks. ### The Backbone DR Architecture The system operates through a three-stage pipeline designed to transition from a broad query to a detailed final document: * **Research Plan Generation:** Upon receiving a query, the agent produces a structured outline of key areas to guide the subsequent information-gathering process. * **Iterative Search Agents:** Two sub-agents work in tandem; one formulates specific search questions based on the plan, while the other performs Retrieval-Augmented Generation (RAG) to synthesize precise answers from available sources. * **Final Report Synthesis:** The agent combines the initial research plan with the accumulated question-answer pairs to produce a coherent, evidence-based final report. ### Component-wise Self-Evolution To ensure high-quality inputs at every stage, the framework employs a self-evolutionary algorithm that optimizes the performance of individual agents: * **Diverse Variant Generation:** The system explores multiple diverse answer variants to cover a larger search space and identify the most valuable information. * **Environmental Feedback:** An "LLM-as-a-judge" assesses these variants using auto-raters for metrics like helpfulness and comprehensiveness, providing specific textual feedback for improvement. * **Revision and Cross-over:** Variants undergo iterative revisions based on feedback before being merged into a single, high-quality output that consolidates the best information from all evolutionary paths. ### Report-level Refinement via Diffusion The core innovation of TTD-DR is modeling the writing process as a denoising diffusion mechanism: * **Messy-to-Polished Transformation:** The framework treats the initial rough draft as a noisy input that requires cleaning through factual verification. * **Denoising with Retrieval:** The agent identifies missing information or weak arguments in the draft and uses search tools as a "denoising step" to inject new facts and strengthen the content. * **Continuous Improvement Loop:** This process repeats in cycles, where each iteration uses newly retrieved information to refine the draft into a more accurate and high-quality final version. TTD-DR demonstrates that shifting AI development from linear generation to iterative, diffusion-based refinement significantly improves the depth and rigor of long-form content. This methodology serves as a powerful blueprint for building autonomous agents capable of handling complex, multi-step knowledge tasks.

AI Agents AI Large Language Models RAG+5

figmaAug 5, 20253 min readCurated summary

Design Systems And AI: Why MCP Servers Are The Unlock | Figma Blog

Design systems provide the shared language AI agents need to generate code that is not merely functional, but consistent with a company’s brand, accessibility standards, and engineering practices. Figma argues that its MCP server unlocks this value by transferring design context—such as components, variables, styles, and code mappings—directly into developer workflows. The result is a feedback loop in which stronger design systems produce better AI output, while AI makes those systems more useful and scalable. ## Design Systems as Context for AI - Design systems already connect design and engineering through: - Shared patterns and terminology - Documentation and best practices - Brand guidelines and reusable code - These same elements give AI agents the context required to produce the “right” output rather than generic interfaces. - A mature design system can therefore become a productivity multiplier for AI-powered product development. - Organizations without a robust system can also use Figma’s MCP server to help implement tokens and components. ## Design Systems as the Shared Language - As AI lowers the barrier between ideas and implementation, product differentiation increasingly depends on craft, visual identity, and user experience. - Design systems help scale that craft while preserving speed, quality, and consistency. - Effective systems provide: - **Scalable foundations:** Tokens for color, spacing, typography, and other design decisions - **Reusable components:** Flexible elements built around a shared source of truth - **Built-in accessibility:** Inclusive experiences by default - They also prevent teams from shipping interchangeable, generic interfaces assembled from common AI-generated parts. ## Why Context Improves AI Code Generation - Figma reports that 68% of developers use AI to write code, but only 32% trust its output. - Without design-system context, AI behaves like a new engineer who has not been onboarded: its code may work, but it may not follow team conventions. - With that context, AI can: - Reuse existing components and patterns - Apply design tokens consistently - Generate higher-quality starting code - Reduce misunderstandings and shorten design-engineering feedback loops ## How Figma’s MCP Server Works - When developers inspect a Figma frame, the MCP server sends relevant context to an AI agent, including: - Components - Styles - Variables - Variable code syntax - **Code Connect** can map design elements to real code resources, allowing agents to use existing implementation libraries. - Even without these mappings, the server supplies styling information that helps agents create more design-informed code. - Automated design-system rule generation can scan a codebase and produce a structured rules file covering: - Token definitions - Component libraries - Style hierarchies - Naming conventions - This file gives AI agents system-level defaults, reducing the need for developers to repeat detailed instructions in every prompt. - Figma MCP also provides annotations that can communicate extra context, including accessibility and interaction behavior. ## The Design-System and AI Flywheel - Better design systems provide richer context to AI agents. - Better context leads to more accurate, on-brand code. - Improved AI output can make design-system adoption and maintenance more valuable. - This creates a reinforcing cycle: robust systems improve AI results, and AI helps teams apply and extend those systems more effectively. Teams seeking reliable AI-generated product code should treat their design system as essential infrastructure and connect it to development tools through mechanisms such as MCP, Code Connect, tokens, rules, and annotations.

Read original

AI Agents Figma Design Systems Model Context Protocol+2

googleJul 31, 2025Original article

MLE-STAR: A state-of-the-art machine learning engineering agent (opens in new tab)

MLE-STAR is a state-of-the-art machine learning engineering agent designed to automate complex ML tasks by treating them as iterative code optimization challenges. Unlike previous agents that rely solely on an LLM’s internal knowledge, MLE-STAR integrates external web searches and targeted ablation studies to pinpoint and refine specific pipeline components. This approach allows the agent to achieve high-performance results, evidenced by its ability to win medals in 63% of Kaggle competitions within the MLE-Bench-Lite benchmark. ## External Knowledge and Targeted Ablation The core of MLE-STAR’s effectiveness lies in its ability to move beyond generic machine learning libraries by incorporating external research and specific performance testing. * The agent uses web search to retrieve task-specific, state-of-the-art models and approaches rather than defaulting to familiar libraries like scikit-learn. * Instead of modifying an entire script at once, the system conducts an ablation study to evaluate the impact of individual pipeline components, such as feature engineering or model selection. * By identifying which code blocks have the most significant impact on performance, the agent can focus its reasoning and optimization efforts where they are most needed. ## Iterative Refinement and Intelligent Ensembling Once the critical components are identified, MLE-STAR employs a specialized refinement process to maximize the effectiveness of the generated solution. * Targeted code blocks undergo iterative refinement based on LLM-suggested plans that incorporate feedback from prior experimental failures and successes. * The agent features a unique ensembling strategy where it proposes multiple candidate solutions and then designs its own method to merge them. * Rather than using simple validation-score voting, the agent iteratively improves the ensemble strategy itself, treating the combination of models as a distinct optimization task. ## Robustness and Safety Verification To ensure the generated code is both functional and reliable for real-world deployment, MLE-STAR incorporates three specialized diagnostic modules. * **Debugging Agent:** Automatically analyzes tracebacks and execution errors in Python scripts to provide iterative corrections. * **Data Leakage Checker:** Reviews the solution script prior to execution to ensure the model does not improperly access test dataset information during the training phase. * **Data Usage Checker:** Analyzes whether the script is utilizing all available data sources, preventing the agent from overlooking complex data formats in favor of simpler files like CSVs. By combining external grounding with a granular, component-based optimization strategy, MLE-STAR represents a significant shift in automated machine learning. For organizations looking to scale their ML workflows, such an agent suggests a future where the role of the engineer shifts from manual coding to high-level supervision of autonomous agents that can navigate the vast landscape of research and data engineering.

AI Agents AI Large Language Models Python+5

lineJul 16, 2025Original article

LY's Tech Conference, ' (opens in new tab)

LY Corporation’s Tech-Verse 2025 conference highlighted the company's strategic pivot toward becoming an AI-centric organization through the "Catalyst One Platform" initiative. By integrating the disparate infrastructures of LINE and Yahoo! JAPAN into a unified private cloud, the company aims to achieve massive cost efficiencies while accelerating the deployment of AI agents across its entire service ecosystem. This transformation focuses on empowering engineers with AI-driven development tools to foster rapid innovation and deliver a seamless, "WOW" experience for global users. ### Infrastructure Integration and the Catalyst One Platform To address the redundancies following the merger of LINE and Yahoo! JAPAN, LY Corporation is consolidating its technical foundations into a single internal ecosystem known as the Catalyst One Platform. * **Private Cloud Advantage:** The company maintains its own private cloud to achieve a four-fold cost reduction compared to public cloud alternatives, managed by a lean team of 700 people supporting 500,000 servers. * **Unified Architecture:** The integration spans several layers, including Infrastructure (Project "DC-Hub"), Cloud (Project "Flava"), and specialized Data and AI platforms. * **Next-Generation Cloud "Flava":** This platform integrates existing services to enhance VM specifications, VPC networking, and high-performance object storage (Ceph and Dragon). * **Information Security:** A dedicated "SafeOps" framework is being implemented to provide governance and security across all integrated services, ensuring a safer environment for user data. ### AI Strategy and Service Agentization A core pillar of LY’s strategy is the "AI Agentization" of all its services, moving beyond simple features to proactive, personalized assistance. * **Scaling GenAI:** Generative AI has already been integrated into 44 different services within the group. * **Personalized Agents:** The company is developing the capacity to generate millions of specialized agents that can be linked together to support the unique needs of individual users. * **Agent Ecosystem:** The goal is to move from a standard platform model to one where every user interaction is mediated by an intelligent agent. ### AI-Driven Development Transformation Beyond user-facing services, LY is fundamentally changing how its engineers work by deploying internal AI development solutions to all staff starting in July. * **Code and Test Automation:** Proof of Concept (PoC) results showed a 96% accuracy rate for "Code Assist" and a 97% reduction in time for "Auto Test" procedures. * **RAG Integration:** The system utilizes Retrieval-Augmented Generation (RAG) to leverage internal company knowledge and guidelines, ensuring high-quality, context-aware development support. * **Efficiency Gains:** By automating repetitive tasks, the company intends for engineers to shift their focus from maintenance to creative service improvement and innovation. The successful integration of these platforms and the aggressive adoption of AI-driven development tools suggest that LY Corporation is positioning itself to be a leader in the "AI-agent" era. For technical organizations, LY's model serves as a case study in how large-scale mergers can leverage private cloud infrastructure to fund and accelerate a company-wide AI transition.

AI Agents AI Generative AI RAG+4

figmaJun 25, 20253 min readCurated summary

6 Skills Every Engineer Needs for the AI Era | Figma Blog

AI should not merely automate engineering work; it should expand how developers explore problems, collaborate, and create better products. Figma argues that engineers who thrive in the AI era will combine technical judgment with user empathy, experimentation, and the ability to direct AI effectively. The goal is to automate repetitive work while preserving—and strengthening—the meaningful parts of product development. ## Use AI for More Than Automation - AI should maximize engineers’ capabilities rather than simply reduce costs. - Developers still need to identify the right problems, understand users, and apply care and craft to their solutions. - Automating tedious tasks frees teams to focus on product meaning, collaboration, and user value. - Employers increasingly value engineers who understand why they are building something, not just how. ## Use Vibe Coding to Explore Possibilities - Vibe coding is presented as a way to explore the problem space, not just generate production code. - Conversational development lets teams test more possible solutions in parallel and quickly produce visual artifacts. - Tools such as Figma Make support rapid prototyping, iteration, and refinement across design and code. - AI-assisted exploration can improve user experience by helping teams consider user needs earlier rather than automating away that consideration. ## Harness Agentic Capabilities - The Model Context Protocol (MCP) enables AI tools such as Cursor and Copilot to communicate with other software. - Figma’s MCP server supplies design context to language models, improving design-informed code generation. - Better context can increase visual fidelity and help developers follow established component libraries and accessibility practices. - Agentic tools are most effective when they have access to the conventions and information that guide the product. ## Audit Your Own Pull Requests - Engineers can use LLMs as a pre-review sounding board before submitting a pull request. - Models familiar with the codebase can identify duplicated implementations, unnecessary rewrites, and other issues. - This self-review improves code quality while reducing the burden on human reviewers. - AI-assisted review can increase engineering throughput without replacing team review. ## Coordinate Teams of AI Agents - Developers are learning to divide complex problems into smaller tasks for multiple AI agents. - They must then evaluate and integrate the agents’ separate solutions. - A key emerging skill is writing detailed Markdown instructions and providing context, much like guiding an intern. - The supplied article ends mid-section, so the sixth skill and the remainder of this discussion are not included. Engineers should treat AI as a partner for exploration, feedback, and coordination—not simply as a code generator. Strong results depend on clear problem framing, relevant context, human judgment, and continued attention to users.

Read original

AI Agents AI Large Language Models Model Context Protocol+2

figmaJun 4, 20252 min readCurated summary

Double Click: What Does MCP Mean for Agentic AI? | Figma Blog

MCP is emerging as a common interoperability layer between AI assistants and external tools or data sources. By allowing models to discover and invoke tools during a conversation, it can make agentic systems faster, more scalable, and less dependent on bespoke integrations. The growing ecosystem—including Figma’s own MCP server—suggests MCP could become foundational infrastructure for the agentic web. ## What MCP Is - Anthropic introduced the Model Context Protocol in November, 2024; support from OpenAI helped drive widespread adoption. - MCP standardizes how assistants such as Claude, Copilot, and Cursor communicate with tools and data. - Developers can avoid building separate custom integrations for every AI assistant and service. - The protocol is compared to: - **USB-C**, as a universal connection for AI applications. - **HTTP**, because it is lightweight, composable, interoperable, and largely unconcerned with the payload. - Microsoft CTO Kevin Scott described MCP as a potential backbone for agent communication and evolution. ## MCP and Agentic AI - MCP gives large language models real-time access to tools and information. - This enables AI agents to act on a user’s behalf rather than merely generate text or code. - The number of available MCP servers is growing rapidly. - Figma created an MCP server that connects design information directly to developer workflows, supporting design-informed code generation. ## MCP as an Accelerant - Traditional API-based workflows often require an LLM to generate code and execute it whenever it needs to interact with a tool. - With MCP, the model can recognize available tools and invoke them directly during a conversation. - This reduces friction and improves speed, efficiency, and scalability. - The resulting experience makes complex actions feel more immediate—users can increasingly “just do things.” ## Early Examples of MCP Use - An automated AI travel agency used four agents across Google Maps, Airbnb, Google Calendar, and Weather. - A Blender MCP server generated a 3D scene of a low-poly dragon guarding treasure from a few natural-language instructions. - Y Combinator hosted a large MCP hackathon, demonstrating the rapid growth of experimentation around the protocol. MCP’s main promise is not simply connecting AI to APIs, but creating a shared, extensible standard that lets agents use many tools dynamically. Its long-term impact will depend on how reliably and responsibly these increasingly capable systems operate.

Read original

AI Agents Figma Large Language Models Model Context Protocol+1