cloudflare·Apr 20, 2026·3 min readCurated summary

Building the agentic cloud: everything we launched during Agents Week 2026

Cloudflare’s Agents Week 2026 introduced a broad set of infrastructure primitives for building and operating AI agents at scale. The company argues that agents require a new cloud model—“Cloud 2.0”—with elastic compute, built-in security, persistent state, specialized tools, and support for agent-driven web traffic. Its announcements span compute environments, identity and networking, developer tooling, inference, voice, email, and memory. ## Compute for Autonomous Agents - **Artifacts** provides Git-compatible, versioned storage for code and data. It supports tens of millions of repositories, remote forking, and access through standard Git clients. - **Cloudflare Sandboxes**, now generally available, give agents persistent isolated computers with shells, filesystems, and background processes. Environments can start on demand and resume where they left off. - **Outbound Workers for Sandboxes** act as programmable, zero-trust egress proxies. They let developers inject credentials and apply dynamic outbound security policies without exposing secrets to agent-generated code. - **Durable Object Facets** allow dynamically generated Workers to create isolated Durable Objects with their own SQLite databases, enabling stateful applications built on the fly. - **Workflows** was rearchitected to support up to 50,000 concurrent executions and a creation rate of 300, making it more suitable for durable, long-running background agents. ## Security, Identity, and Private Networking - **Cloudflare Mesh** provides private network access for users, infrastructure, Workers, and autonomous agents. Combined with Workers VPC, it enables scoped access to private databases and APIs without manually configured tunnels. - **Managed OAuth for Cloudflare Access** lets agents authenticate to internal applications on behalf of users using RFC 9728 rather than insecure shared service accounts. - New identity controls include scannable API tokens, improved OAuth visibility, and resource-scoped permissions to support least-privilege access and automated credential protection. - Cloudflare outlined an enterprise architecture for governing **MCP** deployments using Access, AI Gateway, and MCP server portals. - **Code Mode** reduces MCP token costs, while new Cloudflare Gateway rules help detect unauthorized or “Shadow MCP” usage. ## The Agent Toolbox - A new preview of the **Agents SDK**, called Project Think, aims to provide a more complete platform for agents that can reason, act, and persist. - An experimental **voice pipeline** supports real-time speech-to-text and text-to-speech over WebSockets, requiring roughly 30 lines of server-side code. - **Cloudflare Email Service** entered public beta, allowing agents to send, receive, and process email as a native communication channel. - Cloudflare’s AI platform is becoming a unified inference layer supporting models from more than 14 providers, including third-party model bindings for Workers and an expanded multimodal catalog. - Cloudflare also described a custom infrastructure stack for serving large language models efficiently on its global network. - **Unweight**, a lossless inference-time compression system, reduces model footprints by up to 22%, improving GPU memory efficiency and potentially lowering inference cost and latency. - **Agent Memory** was introduced as a managed service for giving agents persistent memory, though the provided article excerpt ends before detailing its full capabilities. Cloudflare’s announcements collectively position Workers and related services as a platform for the agentic cloud: one capable of running agents, securing their access, preserving their state, and supplying the models and communication tools they need to operate continuously at Internet scale.

Read original(opens in new tab)

InferenceCloudflareDocker & ContainersServerless+4

google·Mar 17, 2025Original article

Generating synthetic data with differentially private LLM inference (opens in new tab)

Researchers at Google have developed an inference-only method for generating differentially private (DP) synthetic data that avoids the high costs and data requirements associated with private fine-tuning. By prompting off-the-shelf large language models (LLMs) with sensitive examples in parallel and aggregating their outputs, the approach can generate thousands of high-quality synthetic data points while maintaining rigorous privacy guarantees. This method allows synthetic data to serve as a secure interface for model development, enabling teams to collaborate without requiring specialized knowledge of differential privacy. ## Differentially Private Prediction and Aggregation The core of this method relies on "private prediction," where privacy is applied to the model's output rather than the model itself. * Sensitive data points are distributed across multiple independent prompts, ensuring that no single individual's record can significantly influence the final output. * The LLM generates next-token predictions for each prompt in parallel, which are then aggregated to mask individual contributions. * The researchers designed a DP token sampling algorithm that treats the standard LLM "softmax" sampling process as a version of the exponential mechanism, a mathematical framework used to select the best option from a set while maintaining privacy. ## Enhancing Efficiency via KV Caching Previous attempts at private prediction were computationally expensive because they required a fresh batch of sensitive examples for every single token generated. * A new privacy analysis allows the system to reuse a fixed batch of sensitive examples across an entire generation sequence. * By maintaining the same context for each generation step, the system becomes compatible with standard inference optimization techniques like KV (Key-Value) caching. * This improvement enables the generation of synthetic data at a scale two to three orders of magnitude larger than prior methods. ## Optimizing Privacy Spend with Public Drafters To preserve the "privacy budget"—the limited amount of information that can be released before privacy is compromised—the method introduces a public drafter model. * The drafter model predicts the next token based solely on previously generated synthetic text, without ever seeing the sensitive data. * Using the sparse vector technique, the system only consumes the privacy budget when the public drafter’s suggestion disagrees with the private aggregate of the sensitive data. * This is particularly useful for structured data, where the drafter can handle formatting and syntax tokens, saving the privacy budget for the actual content. By leveraging off-the-shelf models like Gemma, this approach provides a scalable way to transform sensitive datasets into useful synthetic versions. These synthetic datasets are high-quality enough to replace real data in downstream machine learning tasks, such as in-context learning or fine-tuning models like BERT, without the risk of leaking individual user information.

InferenceAILarge Language ModelsMachine Learning+5