cloudflare-workers-ai

Unifying Workers AI and AI Gateway into a single AI control plane (opens in new tab)

AI Gateway and Workers AI are converging into a unified control plane for accessing models across Cloudflare and external providers. A single Workers binding or REST API can now provide inference, observability, logging, security, and billing without requiring users to choose a product upfront. Cloudflare’s longer-term goal is model-first routing, where applications request capabilities or models while the gateway handles provider selection, failover, and load balancing. ## Unified Bindings and API - The Workers AI binding and AI Gateway now share the same entrypoint. - Requests can use the built-in `default` gateway or a named gateway for separate applications and customized policies. - The unified REST API routes requests through `/ai/` endpoints, using the `cf-aig-gateway-id` header. - This removes the need to decide between Workers AI and AI Gateway before building an application. ## Automatic Observability for Workers AI - Passing `default` as the gateway ID automatically creates an AI Gateway on the first authenticated request. - Requests receive built-in: - Full request and response logging - Token tracking by model - Cost attribution - Latency and error metrics - Developers can begin with the default gateway and later switch to a named gateway for features such as custom caching or application-specific traffic separation. - The AI Gateway dashboard provides detailed visibility into prompts, responses, latency, token usage, and failures. ## Unified Billing with AI Gateway Credits - AI Gateway credits can now pay for Workers AI usage in addition to providers such as OpenAI and Anthropic. - Users can maintain one prepaid credit balance across supported providers. - Workers AI users who use unified billing receive elevated rate limits, subject to current Cloudflare policies and documentation. ## Model-First Routing - Cloudflare plans to route requests based on the desired model rather than requiring users to select a specific provider. - The gateway could handle: - Provider selection - Failover - Load balancing - Capacity management - For example, a request for a model such as Kimi K2.7 Code could be served by Workers AI, the model’s original provider, or another vetted provider hosting the same weights. - Applications could remain available if one provider is overloaded or unavailable. - Users will still be able to restrict traffic to a single provider when necessary. - Routing is intended to preserve requirements such as Zero Data Retention and maintain model quality. Cloudflare recommends using the unified binding or REST API with the default gateway to gain observability and centralized billing immediately. As model-first routing develops, applications can rely less on provider-specific infrastructure and gain greater resilience through automatic provider management.

Growing the Cloudflare AI team with talent from Ensemble AI (opens in new tab)

Cloudflare is bringing key members of Ensemble AI onto its team to improve AI infrastructure and inference efficiency. Ensemble’s work on model compression, structured neural architectures, and parameter-efficient fine-tuning complements Cloudflare’s Workers AI platform. The combined effort aims to make powerful AI models faster, cheaper, and easier to deploy globally. ## Incorporating Ensemble AI’s Expertise - Ensemble AI has focused on reducing the memory, compute, and deployment costs of large language and multimodal models. - Its NdLinear technology replaces standard transformer linear layers while preserving multidimensional structure such as attention heads, channels, and spatial dimensions. - NdLinear-LoRA reduces the number of trainable parameters needed to fine-tune large models. - These techniques complement quantization and vector quantization to improve model efficiency without significantly sacrificing quality. ## Improving AI Inference Economics - Cloudflare Workers AI provides serverless GPU-powered inference across Cloudflare’s global network. - Lower model size, memory usage, and compute requirements can improve throughput, GPU utilization, and overall inference costs. - These improvements are increasingly important for agents, multimodal applications, personalization, fine-tuning, retrieval, and reinforcement learning. - The Ensemble team will contribute to Cloudflare’s existing work, including the Infire inference engine, Unweight tensor compression, and systems for running very large language models. ## Supporting Next-Generation Workloads - Developers increasingly need AI infrastructure that is reliable, affordable, globally distributed, and close to end users—not merely access to models. - Cloudflare’s network, serverless platform, and Workers AI provide a foundation for deploying AI with less operational complexity. - Combining Cloudflare’s infrastructure with Ensemble’s efficient model architectures should enable lower-cost, higher-performance AI deployments at scale. Cloudflare’s stated goal is to make advanced AI workloads more accessible by improving the economics and efficiency of inference across its platform.