Providers Overview

​

Configuring providers without env vars

​

Routing a provider through a proxy

providers:
  openai:
    type: openai
    api_key: "${OPENAI_API_KEY}"
    proxy_url: "socks5://user:password@proxy.internal:1080"

  openai-direct:
    type: openai
    api_key: "${OPENAI_API_KEY_DIRECT}"

export OPENAI_PROXY_URL="http://proxy.internal:3128"
export OPENAI_EU_PROXY_URL="socks5h://eu-proxy.internal:1080"

​

Supported providers

​

Provider notes

• llm-d — point LLMD_BASE_URL at the Router/EPP service and declare LLMD_MODELS when the route does not serve /v1/models. The dedicated provider injects trusted inference-objective and user-path fairness headers.
• Z.ai GLM Coding Plan — set ZAI_BASE_URL=https://api.z.ai/api/coding/paas/v4.
• MiniMax regions and current text models — the default global endpoint is https://api.minimax.io/v1; accounts on the China platform should set MINIMAX_BASE_URL=https://api.minimaxi.com/v1. The current text model IDs are MiniMax-M3 (1,000,000-token context; text, image, and video input) and MiniMax-M2.7 (204,800-token context; text input). Per-million-token USD pricing is $0.60 input,$2.40 output, and $0.12 cached input for M3, and$0.30 input, $1.20 output,$0.06 cached input, and $0.375 cache writes for M2.7. M3 supports adaptive or disabled thinking; M2.7 always thinks.
• Amazon Bedrock Mantle — GPT-5.6 Sol, Terra, and Luna accept only the Responses API on Mantle. GoModel sends /v1/responses requests directly to AWS and selects the required /openai/v1/responses upstream path.
• Fireworks AI — model IDs are account-scoped paths such as accounts/fireworks/models/gpt-oss-120b; use them verbatim in requests and in FIREWORKS_MODELS.
• ElevenLabs — a voice-only provider: /v1/audio/speech (text-to-speech) and /v1/audio/transcriptions (speech-to-text) are supported, but chat, /v1/responses, and embeddings are not (the ❌s above reflect that, not a gateway limitation). The OpenAI voice field must be an ElevenLabs voice_id. See the ElevenLabs guide for supported audio formats and speech-to-text models.
• Chutes AI — defaults to https://llm.chutes.ai/v1 and discovers its current model IDs, context limits, capabilities, and pricing from the live catalog. GoModel translates /v1/responses requests to chat completions; Chutes’ shared LLM endpoint does not expose embeddings. Passthrough support requires explicit operator opt-in by adding chutes to ENABLED_PASSTHROUGH_PROVIDERS.
• Meta (Muse Spark) — the Meta Model API is OpenAI-compatible; set META_API_KEY and route to muse-spark-1.1. Muse Spark models are not in the upstream model catalog yet, so declare context_window and pricing under providers.meta.models metadata in config.yaml if you want cost tracking and context-window reporting.
• Kilo AI — model IDs use provider/model (for example, anthropic/claude-sonnet-4.5) and are forwarded unchanged. GoModel serves /v1/responses by translating it to Kilo’s chat-completions endpoint. If another configured provider exposes the same raw model ID, select Kilo explicitly with kilo/anthropic/claude-sonnet-4.5; GoModel removes only the outer kilo/ routing qualifier before forwarding.
• Xiaomi MiMo — TTS (mimo-v2.5-tts*) and ASR (mimo-v2.5-asr) are served through /v1/audio/speech and /v1/audio/transcriptions (translated to MiMo’s chat-completions audio dialect) as well as directly via chat completions; for 1M context append [1m] to the model ID and list it in XIAOMI_MODELS.
• OpenCode Go (OpenCode Zen) — routes per model: most models use OpenAI-style /chat/completions, while /messages-only models (default qwen3.7-max, override with OPENCODE_GO_MESSAGES_MODELS) are sent to the Anthropic-native endpoint. Chat requests without a reasoning parameter get reasoning_effort: "low" injected, because some models always think and reject requests that omit it (override with OPENCODE_GO_DEFAULT_REASONING_EFFORT). Set OPENCODE_GO_API_KEY; the base URL defaults to https://opencode.ai/zen/go/v1.
• ChatGPT subscription — serves /v1/responses only, billed against the ChatGPT plan’s quota rather than API credit. The upstream accepts a strict parameter allowlist and streams only; GoModel adapts requests and collapses the stream for non-streaming callers. Set CHATGPT_API_KEY to the access token from codex login. Reported cost is not real spend: these model IDs also exist on the OpenAI Platform, so the registry attaches their per-token prices to flat-rate subscription traffic — keep budgets and cost load balancing off it.
• Kimi Code — no per-token pricing; quota refreshes weekly and is also constrained by a rolling 5-hour window. Usage-cost tracking reports zero for Kimi Code, so cost load-balancing cannot price it; prefer conservative retry strategies.
• Hetzner (experimental) — Hetzner declares the inference API experimental: expect breaking changes and no SLA. The model catalogue and rate-limit windows change while the experiment runs; query the live /v1/models endpoint and the official docs instead of relying on snapshots. The example model ID in the table above (Qwen/Qwen3.6-35B-A3B-FP8) is the entry from the official Hetzner docs as of 2026-08-17 and may differ at read time. Free while experimental, so usage-cost tracking reports zero and cost load-balancing cannot price it. No embeddings endpoint; chat, /v1/responses (via chat), model listing, and passthrough only. Passthrough is a generic forwarder — the ✅ marks adapter capability; Hetzner’s tolerance for arbitrary upstream paths is unverified while the API is experimental. hetzner is included in the default ENABLED_PASSTHROUGH_PROVIDERS allowlist so /p/hetzner/... routes work without operator opt-in; remove it from the list to gate them.
• Configured model lists — available for every provider with <PROVIDER>_MODELS, for example OPENROUTER_MODELS=openai/gpt-oss-120b,anthropic/claude-sonnet-4 or ORACLE_MODELS=openai.gpt-oss-120b,xai.grok-3. DeepSeek defaults to https://api.deepseek.com; set DEEPSEEK_BASE_URL only when using a compatible proxy or alternate DeepSeek endpoint. By default, CONFIGURED_PROVIDER_MODELS_MODE=fallback uses those lists only when upstream /models is unavailable or empty. Set CONFIGURED_PROVIDER_MODELS_MODE=allowlist to expose only configured models for providers that define a list, skipping their upstream /models calls.
• Model filters — narrow a large catalog by glob pattern or price instead of listing models one by one: OPENROUTER_MODEL_FILTER_INCLUDE=*:free keeps only OpenRouter’s free tier, and <PROVIDER>_MODEL_FILTER_MAX_PRICE_PER_MTOK caps what a provider may route to. See Filtering a provider’s models.
• Model categories (chat vs embeddings vs audio) — a model’s category comes from its modes, resolved from four sources in precedence order: the remote model registry, operator metadata declared under providers.<name>.models in config.yaml (e.g. modes: [embedding]), capability signals in the provider’s own model listing (Gemini supportedGenerationMethods, Cohere endpoints, OpenRouter architecture modalities, Ollama /api/show capabilities), and finally a conservative name check for models still unclassified: IDs containing embed or matching well-known embedding families (bge, e5, gte, minilm) are categorized as embedding models, and IDs containing rerank as reranking models — namespaced IDs like org/model are checked by their final path segment. Declare operator metadata only when a model stays unclassified after all of this. Categories affect dashboard grouping only; /v1/embeddings routes to any model the provider serves regardless of category.
• vLLM — set VLLM_API_KEY only if the upstream server was started with --api-key.
• llama.cpp / LM Studio — LLAMACPP_BASE_URL is required (llama-server’s default port collides with GoModel’s own 8080, so there is no default); LLAMACPP_API_KEY is optional. Do not register these servers as ollama, which speaks Ollama’s native API. See llama.cpp.
• llm-d — LLMD_BASE_URL is required. LLMD_API_KEY is optional and is sent only when the Gateway in front of the Router requires bearer auth.
• SGLang — set SGLANG_API_KEY only if the upstream server was started with --api-key; otherwise SGLANG_BASE_URL is enough.
• Multiple API keys for one provider — set OPENAI_API_KEY_2, OPENAI_API_KEY_3, and so on to spread sessions across keys while keeping each conversation on one key for prompt-cache affinity. See API key rotation.
• Multiple instances of one provider type — without config.yaml, use suffixed env vars such as OPENAI_EAST_API_KEY and OPENAI_EAST_BASE_URL; add OPENAI_EAST_MODELS to configure that instance’s model list. This registers provider openai-east with type openai. Vertex AI follows the same suffix pattern — VERTEX_US_PROJECT registers provider vertex-us. Vertex project and location env vars must match the instance prefix: for a suffixed instance such as VERTEX_US_PROJECT, also set VERTEX_US_LOCATION and any other suffixed settings for that instance, rather than the generic VERTEX_PROJECT / VERTEX_LOCATION. VERTEX_AUTH_TYPE defaults to Application Default Credentials (gcp_adc).

​

Why some providers have dedicated pages

• Google Vertex AI — needs a GCP project, region, and either Application Default Credentials or a service-account JSON key.
• Amazon Bedrock — no API key of its own; uses the AWS credential chain and requires explicit model access in the Bedrock console.
• Azure OpenAI — deployment-scoped base URLs, the api-version query parameter, and the api-key header instead of Authorization: Bearer.
• Oracle GenAI — requires an OCI IAM policy for generativeaiapikey and a region-specific OpenAI-compatible endpoint URL.
• Google Gemini (AI Studio) — two routing modes (native generateContent vs OpenAI-compatible) with different image-input behavior.
• Anthropic — reasoning effort maps to Claude’s adaptive thinking and effort control, which differ across model generations.
• DeepSeek — reasoning effort mapping quirks for DeepSeek V4.
• Ollama / SGLang / vLLM — local-model hosting with optional multi-instance setup through suffixed env vars and provider-qualified model IDs.
• Xiaomi MiMo — thinking mode on by default, a [1m] context suffix, and TTS/ASR that run through chat completions rather than native audio endpoints.
• OpenCode Go — OpenCode Zen splits models across OpenAI-style /chat/completions and Anthropic-native /messages; GoModel routes per model (the /messages-only set is maintained manually, default qwen3.7-max).
• ChatGPT subscription — a Responses-only upstream with a strict parameter allowlist, used to put GoModel between Codex and a ChatGPT plan.

​

Provider status

• Recent Requests — requests and errors through the gateway in the last 10 minutes.
• Breaker State — the provider’s live circuit-breaker state (see Resilience).
• Models (Recent Traffic) — per-model request/error counts. A model with at least 3 errors making up half or more of its windowed requests is flagged in red — this catches a single broken model on an otherwise healthy provider, which model discovery alone cannot see. Hover a row for the model’s latest error message.