Together and Groq are providers. OpenRouter is a gateway: one OpenAI-compatible endpoint, many upstream models (frontier + open hosts, sometimes Groq/Cerebras/Together themselves). You use it to switch SKUs without rewriting Vol. 18 OpenAI SDK code—and to fail over when a provider 429s. It is not a Model Hub, not a search engine (Perplexity), and not a substitute for DPAs with the actual model vendor when legal requires them.
Hardware peer still worth buying direct: Cerebras. Catalog context: Vol. 11.5 families and Vol. 22.1 frontier providers.
Learning Objectives
By the end of this lesson, students should be able to:
- Define OpenRouter as a multi-provider LLM gateway with OpenAI-compatible APIs.
- Call it by changing
base_url+ model id (oftenprovider/model). - Contrast gateway vs direct Together/Groq/OpenAI on latency, price opacity, and compliance.
- Design fallbacks (primary SKU → cheaper/faster SKU) without pretending routing is magic quality.
- Know when to go direct to a provider for SLA, data processing, and invoices.
- Log provider + model + usage on every call for Vol. 19 token/cost eval.
OpenRouter is a hosted API gateway that accepts OpenAI-style chat/completions (and related) requests and routes them to upstream model providers. You hold one OpenRouter key; OpenRouter holds (or proxies) upstream credentials. Model ids typically encode vendor + model. Optional routing/fallback features exist—treat them as ops, not as an eval strategy. OpenRouter does not train your weights and does not replace wrap-as-data or tool authorization.
Why Gateways Exist
Model SKUs die, rate-limit, or spike in price. A gateway lets product code depend on an interface while config points at anthropic/… today and openai/… tomorrow. The cost is another hop, another ToS, and sometimes less transparent upstream data handling. Serious enterprises often prototype on OpenRouter and pin direct for production SLAs—or the reverse: direct for one hero model, OpenRouter for long-tail experiments.
Pricing / Strengths / Weaknesses (Qualitative)
| Dimension | OpenRouter | Direct OpenAI/Anthropic | Direct Together/Groq | Self-host |
|---|---|---|---|---|
| Pricing posture | Passthrough-ish upstream rates plus gateway margin/credits—confirm live catalog; never assume “cheaper than direct” | Vendor list prices + commits | Open-model token tables | GPU + eng |
| Strengths | One integration; huge SKU catalog; easy A/B; fallbacks; great for prototypes and multi-model products | First-party SLA, features, DPA | Hardware/speed specialists | Residency |
| Weaknesses | Extra hop; compliance/DPA complexity; price/latency variance by upstream; routing can hide failures; not a search/RAG engine | One vendor lock-in per client | Narrower closed-model access | Ops |
Prototype
- One key, many models
- Compare Llama vs Claude vs GPT
- Fine for internal tools
Production hybrid
- Hero model direct
- OpenRouter for spillover/A/B
- Explicit fallback map in config
Avoid
- “Whatever is cheapest” with no eval
- PII to random upstreams
- Gateway as authorization oracle
Do
- Pin allowlisted model ids
- Log upstream provider in traces
- Read OpenRouter + upstream ToS
- Keep FastAPI as the user-facing API
Don’t
- Expose OpenRouter keys to browsers
- Assume GDPR processing is identical to direct OpenAI
- Invent $/M token tables here
- Use OpenRouter as Perplexity search
Python: One Client, Many SKUs
Same Vol. 18 OpenAI client. Model strings look like openai/gpt-4.1-mini or meta-llama/llama-3.1-70b-instruct—confirm current catalog. Optional extra_headers / provider preferences exist in docs; do not depend on undocumented fields.
Related Lectures
| Lecture | Role |
|---|---|
| OpenAI SDK / Anthropic SDK / FastAPI | Direct vs wrapped clients |
| OpenAI / Together / Groq | Upstreams |
| Token usage / latency | Meter the hop |
| Privacy / compliance | Who processes data |
| Cerebras | Next: wafer-scale inference |
“OpenRouter is a model.” It is a router. Second: one key means one DPA covers every upstream. Third: automatic routing maximizes quality. Fourth: OpenRouter is cheaper than every direct vendor always. Fifth: it replaces Vol. 14 RAG. Sixth: it is the Perplexity answer engine.
Knowledge Check
- Short Answer: What is OpenRouter? Answer: A multi-provider LLM API gateway (OpenAI-compatible), not a foundation model.
- True/False: Using OpenRouter automatically satisfies every upstream DPA. Answer: False.
- Multiple Choice: Typical client change is: (a)
base_url+ model id, (b) rewrite in CUDA, (c) Suno credits. Answer: (a). - Short Answer: Name one qualitative weakness vs calling OpenAI directly. Answer: Extra hop, DPA complexity, price/latency variance, or less first-party SLA (any valid).
- True/False: Fallbacks should be allowlisted SKUs in config, not “whatever is cheapest.” Answer: True.
- Multiple Choice: Log for cost/eval: (a) provider + model + usage, (b) only HTTP 200, (c) only GPU TFLOPS. Answer: (a).
- Short Answer: Which Vol. 18 lecture is the native client OpenRouter imitates? Answer: OpenAI SDK.
- True/False: Invent OpenRouter margin percentages from this lecture. Answer: False.
- Multiple Choice: Search-grounded answers belong with: (a) Perplexity / your RAG, (b) OpenRouter routing alone, (c) Udio. Answer: (a).
- Short Answer: Name one upstream you might also call direct. Answer: OpenAI, Anthropic, Together, Groq, or Cerebras (any valid).
Key Takeaways
- OpenRouter = one OpenAI-shaped door to many providers; still just a hop.
- Great for prototypes, A/B, and failovers; compliance may demand direct.
- Allowlist models; log provider; keep FastAPI in front.
- No fake price tables; read the live catalog.
- Next: Cerebras—wafer-scale inference cloud.
Lab: Config file with primary + fallback model ids. Students implement try/except failover and a trace log line provider, model, prompt_tokens, completion_tokens. If no OpenRouter key, mock two upstreams. Grade: no browser keys, written note on when they would sign a direct DPA instead.
Recap: OpenRouter multiplexes vendors behind one SDK. Use it deliberately. Wafer-scale speed next: Cerebras.