OpenRouter routes AI requests Offers key features and benefits * Simplifies AI integration
OpenRouter sits between your application and a collection of large‑language‑model (LLM) providers, acting like a traffic cop that decides which model should handle each request. You send a single HTTP call to OpenRouter; it inspects the payload, consults your routing rules, and forwards the query to the chosen provider (e.g., OpenAI, Anthropic, or a self‑hosted model). The response is then passed back unchanged, so your code only ever talks to one endpoint.
+-----------+ HTTP POST +------------+ API Call +-----------------+
| Your App | ──────────────────► | OpenRouter | ─────────────────► | Provider (LLM) |
+-----------+ +------------+ +-----------------+
▲ │ │
│ ▼ ▼
└───────────────────── Response ────────────────────────────────────┘The routing logic can be static (always use a specific provider) or dynamic (pick the cheapest, fastest, or most capable model based on token cost, latency, or content policy). OpenRouter also normalises authentication headers, so you never embed multiple API keys in your source.
| Feature | What it does |
|---|---|
| Unified endpoint | One URL for all LLMs |
| Policy routing | Choose models by cost, latency, or safety |
| Rate‑limit shim | Consolidates provider limits into a single quota |
| Usage logging | Centralised metrics for billing and debugging |
When your app calls OpenRouter, the platform decides in milliseconds which model should answer. The request lands on a thin HTTP endpoint, where a router engine parses the JSON payload, extracts metadata (model hint, token budget, latency target), and matches it against the routing table you defined. If the primary provider is unavailable or exceeds the cost ceiling, the engine falls back to the next eligible service, guaranteeing delivery without manual retries.
The routing table is a simple YAML file; each rule lists a selector and an ordered list of providers:
# openrouter-routes.yml
- selector: "temperature > 0.7"
providers:
- anthropic/claude-v2
- openai/gpt-4o
- selector: "max_tokens <= 200"
providers:
- openai/gpt-3.5-turbo
- local/llama-2-7bThe engine evaluates selectors in order, picks the first matching rule, and then iterates through its provider list until a successful HTTP 200 response is received.
+-----------+ +-------------------+ +-------------------+
| Client | ---> | OpenRouter API | ---> | Provider A |
+-----------+ +-------------------+ +-------------------+
| ^ fallback
v |
+-------------------+
| Provider B |
+-------------------+Key decisions—cost, latency, model capabilities—are encoded in the selectors, so you can steer traffic toward cheaper or faster models during peak load, and automatically switch to higher‑quality providers for critical queries. This deterministic, rule‑driven approach eliminates ad‑hoc endpoint juggling and keeps your codebase tidy.
OpenRouter’s routing engine is the heart of the service, and it brings three practical capabilities that most developers need. First, it lets you define per‑request rules—model preference, cost ceiling, latency budget—so the platform can pick the cheapest or fastest model that still meets your quality bar. Second, it provides automatic failover: if the primary provider spikes in latency or hits a rate limit, OpenRouter silently retries with a backup without any code change. Third, it aggregates usage metrics across providers, giving you a single dashboard to monitor spend, token counts, and error rates.
{
"model_hint": "gpt-4o-mini",
"max_cost_usd": 0.02,
"latency_ms": 300,
"fallback": ["anthropic/claude-3-haiku", "local/llama-2"]
}The diagram below illustrates the data path:
+-----------+ HTTP POST +------------+ HTTPS +-----------------+
| Client | ──────────────────► | OpenRouter | ───────────────► | LLM Provider(s) |
+-----------+ +------------+ +-----------------+
│ │ │
│ 1. Validate & enrich payload │ │
│ 2. Apply routing rules │ │
│ 3. Forward request │ │
│ │ 4. Return response │
└───────────────────────────────┘─────────────────────────────────┘Because the routing logic lives in a single, stateless endpoint, you can swap providers or adjust budgets without redeploying your app. The built‑in metrics also expose hidden costs—e.g., a model that looks cheap per token may incur higher latency penalties—so you can make data‑driven decisions rather than guessing. In practice, teams report up to 30 % reduction in API spend and a noticeable drop in timeout errors after moving to OpenRouter.
The diagram below shows the minimal pieces that make OpenRouter work, and how data flows between them.
+-----------+ HTTPS POST +-------------+ gRPC/REST +-----------------+
| Client | ───────────────────► | API Layer | ─────────────────► | Router Engine |
+-----------+ +-------------+ +-----------------+
│ │
│ (rules, cost, latency) │
▼ ▼
+----------------+ +-----------------+
| Cache / DB | ◄─────────────| Provider Hub |
+----------------+ fallback +-----------------+
│ │
+--------------------+--------------------+------------+------------+
│ │ │ │
+----------▼----------+ +-------▼-------+ +--------▼--------+ +------------▼-----------+
| OpenAI (GPT‑4) | | Anthropic (Claude) | | Cohere (Command) | | Self‑hosted (vLLM) |
+---------------------+ +-------------------+ +-------------------+ +----------------------+Client – any service that can send an HTTP POST with a JSON payload. API Layer – validates the request, authenticates the key, and throttles traffic. Router Engine – core decision maker; it looks up the routing table, checks the cache for recent results, and selects a provider based on the rules you set (cost ceiling, latency target, model capability). Cache / DB – stores recent completions and routing metadata to avoid duplicate calls and to provide quick fallback paths. * Provider Hub – abstracts each vendor behind a uniform interface, handling authentication, request translation, and error normalisation.
A typical routing rule expressed in JSON looks like this:
{
"model": "gpt-4o-mini",
"max_cost_usd": 0.002,
"latency_ms": 150,
"fallback": ["anthropic/claude-3-haiku", "selfhosted/vllm"]
}When the engine evaluates a request, it first checks the cache, then runs the rule against the live provider status, and finally forwards the payload to the chosen endpoint. The response travels the reverse path, allowing the client to see a single, consistent API regardless of which LLM actually answered.
Developers reach for OpenRouter whenever they need a single point of control over multiple LLM back‑ends. In a micro‑service that generates user‑specific content, the router can automatically select the cheapest model for bulk drafts while switching to a higher‑quality provider for final polishing. A chatbot platform can enforce a latency ceiling—routing to a locally hosted model when the cloud provider’s response time spikes, then falling back to the cloud when the request exceeds the local model’s token limit. Data‑privacy‑sensitive pipelines use OpenRouter to keep personally identifiable information (PII) inside a self‑hosted model, only sending anonymised queries to external APIs.
# Example routing rule (YAML)
rules:
- condition: request.type == "draft"
model: "openai/gpt-3.5-turbo"
max_cost: 0.001
- condition: request.sensitivity == "high"
model: "local/llama-2-7b"
max_latency_ms: 150The diagram below shows the decision flow for a typical request:
+-------------------+
| Application Code |
+--------+----------+
|
v
+--------+----------+
| OpenRouter API |
+--------+----------+
|
+-----+-----+
| Router |
+-----+-----+
| | |
v v v
+---+ +---+ +---+
|A | |B | |C |
|Open| |Anth| |Local|
|AI | |ropic| |LLM |
+---+ +---+ +---+
\ | /
\ v /
+--------+
| Response|
+--------+Typical deployments therefore fall into three buckets: cost‑optimised batch processing, latency‑critical real‑time assistants, and compliance‑driven private inference. By codifying these patterns in routing rules, teams avoid hard‑coding provider switches and gain a single, auditable surface for budgeting, monitoring, and failover.
First, create an account at openrouter.ai and grab the personal API key from the dashboard. Treat it like any other secret: store it in a vault or an environment variable (OPENROUTER_API_KEY).
Next, decide which providers you want to use. In the “Providers” tab you can enable OpenAI, Anthropic, Cohere, or any self‑hosted endpoint you’ve added. Each entry shows its cost per 1 k tokens and a latency SLA; use these numbers to shape your routing rules.
Define a simple routing table in a JSON file. The example below prefers Claude‑2 for creative tasks, falls back to GPT‑4o for speed, and caps daily spend at $5:
{
"rules": [
{ "model_hint": "creative", "primary": "anthropic/claude-2", "fallback": "openai/gpt-4o" },
{ "model_hint": "default", "primary": "openai/gpt-4o", "fallback": "cohere/command" }
],
"budget_usd": 5.00
}Upload the file via the UI or curl it to the /v1/routing endpoint:
curl -X POST https://api.openrouter.ai/v1/routing \
-H "Authorization: Bearer $OPENROUTER_API_KEY" \
-H "Content-Type: application/json" \
--data @routing.jsonNow you can hit the single OpenRouter endpoint from any language. A minimal Python snippet looks like this:
import os, requests, json
payload = {"model_hint":"creative","messages":[{"role":"user","content":"Write a haiku"}]}
resp = requests.post(
"https://api.openrouter.ai/v1/chat/completions",
headers={"Authorization": f"Bearer {os.getenv('OPENROUTER_API_KEY')}"},
json=payload,
)
print(resp.json()["choices"][0]["message"]["content"])The flow is straightforward, as illustrated below:
+-----------+ +--------------+ +-----------------+
| Your App | ---> | OpenRouter | ---> | Selected Provider|
+-----------+ +--------------+ +-----------------+
^ | ^ |
| v | v
Env vars Routing Provider API
(API key) Engine (OpenAI, Anthropic,…)What does OpenRouter actually do? It receives a single HTTP POST, looks at the payload’s metadata, and forwards the call to the LLM that best satisfies your rules—whether that’s the cheapest model, the fastest one, or a specific provider you trust.
Do I need to know every provider’s API? No. OpenRouter normalises the request and response formats, so you write once and let the router handle the quirks of OpenAI, Anthropic, Cohere, etc.
How are routing rules defined? You supply a JSON object with fields like model_hint, max_cost, and latency_target. The router engine matches these against a table you configure in the dashboard or via a simple CLI.
What happens if the primary model fails? The engine automatically falls back to the next eligible provider, preserving the original latency and cost constraints. No code changes are required.
Is there a way to see the decision path? Yes—enable tracing and you’ll get a tiny log entry showing which provider was chosen and why.
# Example CLI to set a routing rule
openrouter rules add \
--model_hint gpt-4o-mini \
--max_cost 0.001 \
--latency_target 150msWhere does the request travel?
+-----------+ +-----------+ +-----------------+
| Your App | ---> | OpenRouter| ---> | Selected LLM API|
+-----------+ +-----------+ +-----------------+
^ | |
| v v
Retry/Failover Routing Engine Provider‑specificCan I limit usage per month? Yes, attach a quota to your API key; once the limit is hit, OpenRouter returns a 429 error, letting you enforce budget caps without extra code.
OpenRouter is a unified AI routing platform that lets developers send a single request to a gateway which then forwards the prompt to the appropriate large‑language‑model (LLM) provider—OpenAI, Anthropic, Cohere, etc. It abstracts away the differences in endpoint URLs, authentication methods, and request formats, making it easier to experiment with multiple models, implement fallback strategies, and manage vendor lock‑in. By handling rate‑limiting, telemetry, and model selection logic centrally, OpenRouter streamlines multi‑model workflows and accelerates prototyping.
Authentication is performed with a bearer token you receive after creating an account on the OpenRouter dashboard. Include the token in the `Authorization: Bearer <YOUR_TOKEN>` header of every HTTP request. The API follows the OpenAI‑compatible JSON schema: send a `POST` to `https://openrouter.ai/api/v1/chat/completions` with `model`, `messages`, and optional parameters like `temperature`. The response mirrors the OpenAI format, so existing client libraries (e.g., `openai` Python package) can be used with minimal code changes by pointing them to the OpenRouter base URL.
Yes. When you call the `/chat/completions` endpoint you specify the target model using the `model` field, which can be any identifier supported by OpenRouter (e.g., `openai/gpt-4o`, `anthropic/claude-3.5-sonnet`). OpenRouter then forwards the request to the corresponding provider, handling any provider‑specific transformations automatically. You can also define routing rules in the dashboard—such as fallback to a cheaper model if the primary model exceeds latency thresholds—by creating a "routing profile" that maps conditions to model identifiers.
Key alternatives include LangChain’s "LLM gateway" which wraps multiple providers behind a common interface, and LlamaIndex’s "ServiceContext" that abstracts model calls. Other options are the open‑source project "OpenAI Proxy" which can be self‑hosted to route to different back‑ends, and cloud‑native solutions like AWS Bedrock or Azure OpenAI that let you switch providers within the same ecosystem. Each alternative varies in ease of setup, vendor lock‑in, and feature set, so choose based on your infrastructure preferences and required model coverage.
OpenRouter uses a consumption‑based pricing model where you pay per token processed on the underlying provider, plus a small platform fee (typically 0.5–1 ¢ per 1 K tokens) that covers routing and telemetry. There is a free tier that grants up to 100 K tokens per month across all models, allowing developers to experiment without cost. Beyond the free quota, you are billed monthly based on usage; pricing details for each supported model are displayed in the dashboard, and you can set hard limits to avoid unexpected spend.