Sandboxing AI agents, 100x faster
Cloudflare argues that AI-generated code needs secure execution, but traditional containers are too slow, memory-intensive, and difficult to scale for consumer-scale agents. Its Dynamic Worker Loader uses lightweight V8 isolates to create disposable, isolated sandboxes in milliseconds, with controlled access to APIs and no internet connectivity. The result is a sandbox roughly 100 times faster and substantially more memory-efficient than containers, provided agents can write JavaScript. ## Why Containers Fall Short - AI-generated code cannot safely run directly through `eval()`, since prompts could cause the model to introduce vulnerabilities. - Containers provide isolation but typically: - Take hundreds of milliseconds to start - Consume hundreds of megabytes of memory - Require warm instances to reduce latency - May encourage unsafe container reuse - These limitations make containers poorly suited to running a fresh sandbox for every request or user agent. ## Dynamic Worker Loader - Cloudflare’s Dynamic Worker Loader lets a Worker instantiate another Worker dynamically from runtime-provided code. - The host can: - Supply generated JavaScript modules - Expose selected APIs through RPC stubs - Disable or intercept outbound internet access - Invoke methods exported by the dynamically loaded Worker - The feature is in open beta for paid Workers users. ## Faster, Smaller Isolates - Dynamic Workers use V8 isolates, the same sandboxing technology underlying Cloudflare Workers. - Isolates: - Start in a few milliseconds - Use only a few megabytes of memory - Are approximately 100 times faster and 10–100 times more memory-efficient than typical containers - A new isolate can be created for one request and discarded afterward without maintaining a pool of warm sandboxes. ## Scalability and Latency - Dynamic Worker Loader has no container-style global concurrency or sandbox-creation limits. - It relies on the infrastructure that already scales Cloudflare Workers to millions of requests per second. - Each request could theoretically load its own isolated sandbox, even at very high concurrency. - Dynamic Workers commonly run on the same machine or thread as their parent Worker, avoiding network round trips and warm-sandbox lookup delays. - They are available across Cloudflare’s global network. ## JavaScript as the Agent Runtime - The main limitation is that agent-generated code should generally be JavaScript. - Workers also support Python and WebAssembly, but JavaScript is faster to load for short-lived snippets. - Cloudflare argues this is acceptable because: - LLMs can generate major programming languages - JavaScript has extensive training data - JavaScript was designed for web-based sandboxed execution ## TypeScript APIs for Agent Tools - Agents still need access to external capabilities such as chat systems and APIs. - TypeScript interfaces provide a concise way to describe these programming APIs. - Compared with MCP’s flat tool schemas or verbose OpenAPI specifications, TypeScript can express: - Methods and parameters - Return types and promises - Objects such as messages - Subscription and disposal behavior - This gives agents precise API knowledge with fewer tokens and lets them write direct code rather than issuing numerous tool calls. Dynamic Worker Loader is presented as a practical foundation for secure, disposable AI-agent execution: use V8 isolates for low-latency sandboxing, expose only narrowly defined TypeScript/RPC capabilities, and block network access unless explicitly required.
Read original(opens in new tab)