Smart Routing

OpenGateway resolves your model ID to a provider and region, tries fallbacks in priority order, and records every attempt in debug.routing.

You send a model ID. OpenGateway decides which provider and region serves it — and, if you configured fallbacks, what to try next when one fails.

How a request is routed#

  1. The owner/model ID resolves to a provider and region.
  2. If you set extra.fallbacks, those models join the attempt list in priority order (capped at three targets including the primary).
  3. On a retryable failure (5xx, timeout, rate limit), the gateway moves to the next target. The first success returns.

Routing happens at the gateway, so it behaves identically from cURL, Python, or any SDK — there is no client-side retry logic to write.

Seeing the path#

Every attempt is recorded. Ask for debug.routing and the response lists each provider / region / status, in the order they were tried.

"routing": [
  { "provider": "openai", "region": "global", "status": "failed" },
  { "provider": "anthropic", "region": "global", "status": "succeeded" }
]

Configure it#

Fallbacks

The extra.fallbacks policy and what counts as a failure.

Debug mode

Surface routing attempts with debug.routing.