techlist.io
KREN
cloudflare

Your AI bill is out of control. Cloudflare can fix it now. (opens in new tab)

ci-cdoauthcloudflare-accessidentity-providerrate-limitingcloudflare-ai-gatewayjson-web-tokenresponse-caching

Cloudflare argues that uncontrolled AI spending stems from shared API keys, poor attribution, and defaulting to the most expensive models. Its AI Gateway now adds dollar-based spend limits, while a closed beta combines Cloudflare Access with identity-based budgets and routing. The goal is to make AI costs visible, attributable, and controllable without disrupting workflows.

The AI Spending Problem

  • Companies encouraged aggressive AI adoption before establishing budgets or monitoring.
  • Shared API keys make it difficult to identify whether costs came from employees, teams, training pipelines, or runaway CI jobs.
  • Without routing rules or spending limits, users often choose frontier models for tasks that could use cheaper alternatives.
  • Visibility and controls are necessary to measure AI ROI and prevent waste.

What AI Gateway Provides

AI Gateway sits between applications and providers such as OpenAI, Anthropic, and Google.

  • Unified billing across providers and models.
  • Centralized logging of requests, token counts, and costs.
  • Response caching and rate limiting.
  • Content guardrails that can block PII and secrets before they reach models.
  • Previously, however, it lacked detailed attribution and budget controls.

Dollar-Based Spend Limits

  • Budgets are measured in dollars rather than tokens and track cumulative request costs in real time.
  • Limits can be scoped by:
    • Model
    • Provider
    • Custom attributes such as user, team, or application
  • Budgets support daily, weekly, or monthly windows, including fixed and rolling periods.
  • When a limit is reached, requests can be blocked by default.
  • Dynamic Routes can instead redirect requests to a cheaper fallback model.
  • Spend limits are available in open beta across all AI Gateway plans through the dashboard or API.

Identity-Driven Attribution and Policies

Cloudflare Access can attach verified identity information to AI Gateway requests.

  • Authenticated user identities are extracted from Access JWTs and added as request metadata.
  • Organizations can track usage by employee, team, identity-provider group, or service.
  • Per-user budgets can give different spending limits to individual contributors and senior engineers.
  • Team policies can control which models different groups may use.
  • Access service tokens provide named identities for CI/CD pipelines and autonomous agents.
  • Logs include details such as email address, IdP group, or service-token name for export to analytics systems.
  • Configuration uses Cloudflare Access applications and existing identity-provider groups, without requiring custom Workers or manual JWT parsing.

Cloudflare’s Internal Use

  • Cloudflare routes millions of requests and billions of tokens through AI Gateway each month.
  • It uses identity metadata to attribute costs, understand team usage, and manage budgets.
  • The company is making this internal approach available through the closed beta for identity-driven budgets and policies.

Organizations should route AI traffic through a gateway, assign verified identities, set dollar-based budgets, and use model fallback rules to balance cost control with developer productivity.