AI Gateway Updates & Release Notes

AI Gateway's REST API now returns consistent responses when an AI provider rejects credentials. The change applies to POST /ai/run ↗︎.

Scenario

Previous AI Gateway response

New AI Gateway response

ElevenLabs

Provider-specific UserCredentialsError with HTTP 403

HTTP 401 with error code 2009

Google Vertex

HTTP 500 for rejected credentials, with upstream retries

HTTP 401 with error code 2009; the request fails without retrying the provider

All other providers

HTTP 402 or another provider-specific status for rejected credentials

HTTP 401 with error code 2009

When using Unified Billing, the provider rejects the credentials

Provider-specific authentication error

HTTP 503

Update applications that handle AI Gateway REST API errors to treat HTTP 401 as an invalid or rejected provider credential.

For details about providing provider credentials, refer to Bring your own provider keys.

Web Search API is now available in beta. Web Search API lets your AI agents and applications search the Internet and ground their responses in live information, instead of guessing URLs or relying on a model's training cutoff.

At launch, you can choose between three search providers: Ceramic.ai, Exa, and Linkup. All three support Zero Data Retention for requests made through Cloudflare, and all have committed to Cloudflare's verified bot crawling standards.

Web Search API runs through AI Gateway, so search requests appear in your gateway logs and are billed to your AI Gateway credits at each provider's list API price, with no additional markup. You can also bring your own provider API key.

Call Web Search API with the REST API:

curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/websearch/ \
  --request POST \
  --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \
  --header "Content-Type: application/json" \
  --data '{
    "query": "What are some fun things to do in Salt Lake City as fall approaches?",
    "provider": "ceramic",
    "limit": 5,
    "options": { "gateway": { "id": "default" } }
  }'

Or from a Worker with the AI binding:

const response = await env.AI.websearch({
	gatewayId: "default", …

AI Gateway now supports Machine Payments in beta. With Machine Payments, clients can use the x402 protocol to pay for eligible inference requests directly from a stablecoin wallet instead of maintaining a prepaid credit balance.

Machine Payments is available for the /ai/run endpoint with select open models. To request x402 payment, authenticate with a Cloudflare API token and include the Cloudflare-specific Payment-Method: x402 header:

curl -iX POST "https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run" \
  --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \
  --header "Payment-Method: x402" \
  --header "Content-Type: application/json" \
  --data '{
    "model": "z-ai/glm-4.7-flash",
    "input": {
      "messages": [
        {
          "role": "user",
          "content": "What is Cloudflare?"
        }
      ]
    }
  }'

An x402-compatible client handles the payment challenge, signs an authorization from the client's wallet, and retries the request. Machine Payments currently requires customers to be based in the United States and have a credit card on file.

For prerequisites, eligible models, and transaction details, refer to Machine Payments (x402).

AI Gateway User Insights now gives you more context about the traffic flowing through your gateway. It shows what users and agents are doing with AI, and where a selected model may be more capable than a task requires.

On the analysis side, User Insights groups conversations by task, tracks conversation turns, and helps you compare model fit with cost and latency.

User Insights task and model analysis grouped by task categories

The Potential Savings view highlights requests that may work with faster or less expensive models without compromising output quality. These are the same signals that Cloudflare's Auto Router uses to select a model based on task and cost.

Potential Savings view comparing tasks and suggested models

These new insights are available to all AI Gateway customers at no additional cost. For more information, refer to User Insights.

AI Gateway can now require credentials for third-party provider requests. Credentials must accompany the request or be stored on the gateway. This setting prevents fallback to Unified Billing with Cloudflare-managed credentials.

Turn on Require provider credentials in your gateway settings. To use the API, set byok_only to true in the request body of a PUT request to update the gateway:

{
	"byok_only": true
}

To require provider credentials for one third-party request, set the cf-aig-no-wholesale header to true. This header cannot relax the gateway setting.

Requests without applicable credentials then return an HTTP 400 response. Workers AI requests remain allowed, and the setting does not change their configured billing mode.

For configuration details and request-level controls, refer to Prevent Unified Billing fallback for BYOK third-party providers.

AI Gateway custom costs now support cache-read and cache-write token rates. This lets custom cost metrics reflect negotiated cache pricing across providers.

Add per_cache_read_token or per_cache_write_token to the cf-aig-custom-cost header:

{
	"per_token_in": 0.000001,
	"per_token_out": 0.000002,
	"per_cache_read_token": 0.0000001,
	"per_cache_write_token": 0.0000005
}

Cache-token pricing activates when either cache rate is present. An omitted cache rate defaults to per_token_in. If both cache rates are omitted, AI Gateway preserves the existing input and output calculation.

Providers can include cache tokens within input tokens or report them separately. AI Gateway automatically accounts for these differences and prevents double-counting.

For more information, refer to Custom costs.

AI Gateway monthly usage invoices, issued at the beginning of each month for the previous month's usage, now show a single total cost for each model. These invoices no longer break out input and output token quantities and unit prices into separate line items. This change does not apply to invoices for AI Gateway credit purchases.

For example, an invoice that previously included these separate line items:

• anthropic claude-haiku-4-5-20251001 Input Tokens: 40,000 tokens at $0.000001 ($0.04)
• anthropic claude-haiku-4-5-20251001 Output Tokens: 24,000 tokens at $0.000005 ($0.12)

The updated invoice includes one line item: anthropic/claude-haiku-4.5: $0.16.

AI Gateway has also standardized model names across invoices and logs. Model variants that previously appeared with provider-specific version suffixes now use a consistent provider/model identifier.

For more information, refer to the Unified Billing documentation and AI Gateway logging documentation.

GPT-5.6 Sol is available through AI Gateway, and for a limited time you can use it at 50% off. If you are already using AI Gateway, point to the openai/gpt-5.6-sol model and the discounted pricing applies automatically — no promo code needed.

The promotion is available for Unified Billing users only (not Bring Your Own Keys). Load credits onto AI Gateway and start sending requests to openai/gpt-5.6-sol.

Discounted pricing during the promotion:

Usage

Promotional price

Standard price

Input

$2.50 per 1M tokens

$5 per 1M tokens

Output

$15 per 1M tokens

$30 per 1M tokens

Cache read

$0.25 per 1M tokens

$0.50 per 1M tokens

The promotion runs through September 18, 2026. After that date, GPT-5.6 Sol requests return to standard pricing.

For more details, refer to the Unified Billing documentation and the GPT-5.6 Sol model page.

Workers AI and AI Gateway now provide a unified path for accessing models and managing inference traffic. Use the same AI binding and REST API to call models hosted on Workers AI or by supported third-party providers, with AI Gateway providing observability, logging, caching, security, and billing controls.

Unified entrypoints and observability

The AI binding supports both Workers AI and third-party models through env.AI.run(). The REST API provides shared /ai/ endpoints with Cloudflare authentication across providers.

Route a Workers AI request through AI Gateway by specifying a gateway ID. Use default to automatically create a gateway on the first authenticated request, or specify an existing gateway to separate applications and workloads:

const response = await env.AI.run(
	"@cf/zai-org/glm-5.2",
	{
		messages: [{ role: "user", content: "What is the capital of France?" }],
	},
	{
		gateway: { id: "default" },
	},
);

const response = await env.AI.run(
	"@cf/zai-org/glm-5.2",
	{
		messages: [{ role: "user", content: "What is the capital of France?" }],
	},
	{
		gateway: { id: "default" },
	},
);
``` …

AI Gateway now includes User Insights, a dashboard that gives you two things at once: clear visibility into how much your organization spends on AI, and a security signal that surfaces users whose usage suddenly looks abnormal. It works on the traffic already flowing through your gateway, so there is no additional setup.

On the spend side, User Insights shows organization-wide totals for cost, requests, tokens, and adoption, and lets you drill into an individual user to see their spend, top models and providers, cache hit rate, and more. To attribute usage to individual users, add a user identifier with custom metadata or put your gateway behind Cloudflare Access.

On the security side, User Insights baselines each user's normal usage from their 95th percentile (p95) session cost over the last 30 days, then flags sessions that exceed both that baseline and an organization-level threshold. A sudden jump above a user's own pattern is often the first sign of a compromised credential or a misbehaving agent, so you can investigate before it shows up on your bill.

User Insights is available to all AI Gateway customers at no additional cost.

AI Gateway now integrates with Cloudflare Access, giving you two new capabilities:

• Protect your gateway endpoint. Put your AI Gateway behind Access so you can set policies that control who is allowed to call a specific gateway's endpoint.
• Identity-aware controls. When traffic reaches AI Gateway through an Access-protected custom domain, AI Gateway can use the authenticated user's Access identity in logs, analytics, routing, and spend controls.

With identity-aware controls, you can set spend limits by authenticated user, control which gateways different users can access, filter logs by user, and build policies without passing user IDs from the client application. AI Gateway adds the verified Access user ID to request metadata as cf.user_id.

For setup instructions, refer to Cloudflare Access.

AI Gateway now supports spend limits — cost-based budgets that track cumulative dollar spend and block requests when the budget is exceeded. Unlike rate limiting, which caps the number of requests, spend limits track actual cost based on token usage and model pricing.

You can scope limits by model, provider, or custom metadata dimensions. For example, give each user a $200/day budget, cap total gateway spend at $10,000/day, or limit a specific model to $50/day per user. Each rule uses a configurable time window with fixed or sliding enforcement.

Spend limits work with both Unified Billing and BYOK requests for models with known pricing.

For more details, refer to the Spend limits documentation.

AI Gateway now uses the AI REST API on api.cloudflare.com. You can call any model — whether from OpenAI, Anthropic, Google, or hosted on Workers AI — through one unified API, using the same endpoints and authentication regardless of provider. Four endpoints are available:

• POST /ai/run — universal endpoint for all models and modalities
• POST /ai/v1/chat/completions — OpenAI SDK compatible
• POST /ai/v1/responses — OpenAI Responses API compatible
• POST /ai/v1/messages — Anthropic SDK compatible

curl -X POST "https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/v1/chat/completions" \
  --header "Authorization: Bearer $CLOUDFLARE_API_TOKEN" \
  --header "Content-Type: application/json" \
  --data '{
    "model": "openai/gpt-5.5",
    "messages": [{"role": "user", "content": "What is Cloudflare?"}]
  }'

All AI Gateway features — logging, caching, rate limiting, and guardrails — are applied automatically. Third-party models are billed through Unified Billing, so you do not need to manage separate provider API keys.

Third-party model requests are routed through your account's default gateway, which is created automatically on first use. To route requests through a specific gateway, add the cf-aig-gateway-id header. …

AI Gateway now supports automatic retries at the gateway level. When an upstream provider returns an error, your gateway retries the request based on the retry policy you configure, without requiring any client-side changes.

You can configure the retry count (up to 5 attempts), the delay between retries (from 100ms to 5 seconds), and the backoff strategy (Constant, Linear, or Exponential). These defaults apply to all requests through the gateway, and per-request headers can override them.

Retry Requests settings in the AI Gateway dashboard

This is particularly useful when you do not control the client making the request and cannot implement retry logic on the caller side. For more complex failover scenarios — such as failing across different providers — use Dynamic Routing.

For more information, refer to Manage gateways.

AI Gateway now supports the cf-aig-collect-log-payload header, which controls whether request and response bodies are stored in logs. By default, this header is set to true and payloads are stored alongside metadata. Set this header to false to skip payload storage while still logging metadata such as token counts, model, provider, status code, cost, and duration.

This is useful when you need usage metrics but do not want to persist sensitive prompt or response data.

curl https://gateway.ai.cloudflare.com/v1/$ACCOUNT_ID/$GATEWAY_ID/openai/chat/completions \
  --header "Authorization: Bearer $TOKEN" \
  --header 'Content-Type: application/json' \
  --header 'cf-aig-collect-log-payload: false' \
  --data '{
    "model": "gpt-4o-mini",
    "messages": [
      {
        "role": "user",
        "content": "What is the email address and phone number of user123?"
      }
    ]
  }'

For more information, refer to Logging.

You can now start using AI Gateway with a single API call — no setup required. Use default as your gateway ID, and AI Gateway creates one for you automatically on the first request.

To try it out, create an API token with AI Gateway - Read, AI Gateway - Edit, and Workers AI - Read permissions, then run:

curl -X POST https://gateway.ai.cloudflare.com/v1/$CLOUDFLARE_ACCOUNT_ID/default/compat/chat/completions \
  --header "cf-aig-authorization: Bearer $CLOUDFLARE_API_TOKEN" \
  --header 'Content-Type: application/json' \
  --data '{
    "model": "workers-ai/@cf/meta/llama-3.3-70b-instruct-fp8-fast",
    "messages": [
      {
        "role": "user",
        "content": "What is Cloudflare?"
      }
    ]
  }'

AI Gateway gives you logging, caching, rate limiting, and access to multiple AI providers through a single endpoint. For more information, refer to Get started.

Workers AI and AI Gateway have received a series of dashboard improvements to help you get started faster and manage your AI workloads more easily.

Navigation and discoverability

AI now has its own top-level section in the Cloudflare dashboard sidebar, so you can find AI features without digging through menus.

AI sidebar navigation in the Cloudflare dashboard The new top-level AI section in the dashboard sidebar.

Onboarding and getting started

Getting started with AI Gateway is now simpler. When you create your first gateway, we now show your gateway's OpenAI-compatible endpoint and step-by-step guidance to help you configure it. The Playground also includes helpful prompts, and usage pages have clear next steps if you have not made any requests yet.

AI Gateway onboarding flow The first-run setup experience for new gateways. …

Cloudflare Secrets Store is now integrated with AI Gateway, allowing you to store, manage, and deploy your AI provider keys in a secure and seamless configuration through Bring Your Own Key ↗︎. Instead of passing your AI provider keys directly in every request header, you can centrally manage each key with Secrets Store and deploy in your gateway configuration using only a reference, rather than passing the value in plain text.

You can now create a secret directly from your AI Gateway in the dashboard ↗︎ by navigating into your gateway -> Provider Keys -> Add.

Import repo or choose template

You can also create your secret with the newly available ai_gateway scope via wrangler ↗︎, the Secrets Store dashboard ↗︎, or the API ↗︎.

Then, pass the key in the request header using its Secrets Store reference:

curl -X POST https://gateway.ai.cloudflare.com/v1/<ACCOUNT_ID>/my-gateway/anthropic/v1/messages \
 --header 'cf-aig-authorization: ANTHROPIC_KEY_1 \
 --header 'anthropic-version: 2023-06-01' \ …

Users can now use an OpenAI Compatible endpoint in AI Gateway to easily switch between providers, while keeping the exact same request and response formats. We're launching now with the chat completions endpoint, with the embeddings endpoint coming up next.

To get started, use the OpenAI compatible chat completions endpoint URL with your own account id and gateway id and switch between providers by changing the model and apiKey parameters.

OpenAI SDK Examplejs

import OpenAI from "openai";
const client = new OpenAI({
	apiKey: "YOUR_PROVIDER_API_KEY", // Provider API key
	baseURL:
		"https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/compat",
});

const response = await client.chat.completions.create({
	model: "google-ai-studio/gemini-2.0-flash",
	messages: [{ role: "user", content: "What is Cloudflare?" }],
});

console.log(response.choices[0].message.content);

Additionally, the OpenAI Compatible endpoint can be combined with our Universal Endpoint to add fallbacks across multiple providers. That means AI Gateway will return every response in the same standardized format, no extra parsing logic required!

Learn more in the OpenAI Compatibility documentation.