OrcaRouter

What is OrcaRouter?

OrcaRouter is a cutting-edge AI router and gateway that acts as a single, intelligent interface to over 200 large language models (LLMs) and AI services. By analyzing each user request (or "prompt"), it dynamically selects the best possible model to handle the task—whether it's a top-tier "frontier" model like GPT-5.5 Pro or a more cost-effective open-source alternative. Its core mission is to deliver enterprise-grade AI performance, reliability, and governance while eliminating token markup, allowing users to pay direct provider prices and save up to 40% on inference costs.

What are the Main Features of OrcaRouter?

1. Intelligent, Adaptive Routing

OrcaRouter's flagship feature is its smart routing engine. It grades every prompt in under 1 millisecond using contextual embeddings and online learning from real traffic. With 75.5% routing accuracy (leading the RouterArena benchmark as of June 2026), it ensures each request is sent to the optimal model based on your chosen strategy: cheapest acceptable quality, highest quality, or a balanced adaptive mode that learns from your usage patterns.

2. Zero Token Markup & Transparent Pricing

The platform operates on a fundamental principle of zero markup. You pay the exact published rate of the underlying model provider (e.g., OpenAI, Anthropic, Google). Every request receipt shows the exact cost, chosen model, and provider, ensuring complete cost transparency without opaque blended rates.

3. Production-Ready Resilience and Failover

OrcaRouter guarantees high availability. If a provider experiences an outage or rate limit, the system automatically fails over to a healthy, equivalent model in under 50 milliseconds—often before the user's request would have timed out. This built-in load balancing protects your application from upstream instability.

4. Comprehensive Governance and Safety

The gateway includes robust guardrails that run pre-billing, such as a PII Shield and content policy enforcement, blocking unsafe requests with a clean 400 error. For the agent era, it features a risk-scored Agent Firewall that grades and controls tool/MCP calls (ALLOW, REVIEW, BLOCK) and includes anomaly detection for cost and rate spikes.

5. Unified Observability and Management

Gain full visibility into every API call with structured logs that detail cost, latency, model choice, and routing rationale. Features like version-controlled prompt management (A/B testing, instant rollbacks) and intelligent caching (billing repeated prompts at lower cache rates) provide powerful control without code deploys.

6. Drop-in API Compatibility

OrcaRouter offers full OpenAI-compatible and MCP server endpoints. You can integrate it by changing just one line of code—the API base_url—in your existing OpenAI, Anthropic, Google, LangChain, or LlamaIndex setup. All existing SDK calls, streaming, and model names continue to work seamlessly.

How to Use OrcaRouter?

Using OrcaRouter is designed to be incredibly simple and requires minimal code changes:

1. Sign Up & Get Your Key: Register for a free account on the OrcaRouter website and obtain your unique API key.

2. Update Your Client Configuration: In your application code, simply change the base_url in your OpenAI client initialization to https://api.orcarouter.ai/v1 and replace your API key with the OrcaRouter key.

   # Before
   client = OpenAI(api_key="sk-openai-...")
   
   # After
   client = OpenAI(
       base_url="https://api.orcarouter.ai/v1",
       api_key="sk-orca-..."
   )
   

3. Make API Calls as Usual: Continue making your client.chat.completions.create() calls as normal. You can use the special model identifier "orcarouter/auto" to let the router pick the best model, or specify any of the 200+ supported models directly.

4. Monitor and Manage: Use the OrcaRouter dashboard to observe traffic, analyze costs, set up routing rules, configure guardrails, and manage prompt versions.

What is the Pricing for OrcaRouter?

OrcaRouter employs a unique pricing model where the core routing service is free. You only pay for the AI models you use, at their direct provider rates.

• Token Costs: $0 markup. You pay exactly what the model provider (OpenAI, Anthropic, etc.) charges per million tokens for input and output.

• Platform Plans:

  • Hacker (Free): Includes smart routing across 200+ models, auto-failover, basic observability, prompt versioning, and 3 API keys. Zero token markup forever.

  • Team ($499/month): Includes all Hacker features, plus up to 10 team seats, advanced compliance tools (GDPR, SOC 2, HIPAA), unlimited API keys, and priority support. Still has zero token markup.

  • Enterprise (Custom): Offers private/on-prem deployment, 99.99% uptime SLA, dedicated infrastructure and support, with custom pricing.

Helpful Tips for Using OrcaRouter

• Start with orcarouter/auto: For most use cases, letting the router automatically select the model is the easiest and most cost-effective way to start. It balances quality and cost effectively.

• Leverage the Caching: Structure repeated prompts (like system instructions) identically to benefit from automatic caching, which can bill at a fraction of the standard input token cost.

• Define Routing Rules: For specialized workloads, use the YAML-based Routing DSL to create custom rules. For example, you can route all high-difficulty agent tasks to Claude Opus and all short queries to the cheapest capable model.

• Use the Observability Dashboard: Regularly check the logs to understand cost drivers, model performance, and routing decisions. This data is invaluable for optimizing your AI spend and performance.

• Set Up Guardrails Early: Proactively configure PII filters and content policies to prevent unwanted charges from blocked requests and to enforce compliance from the start.

Frequently Asked Questions (FAQ)

What is an AI router?

An AI router is a system that sits between your application and multiple AI models. It intelligently analyzes each request (prompt) and directs it to the most suitable model from a vast pool, optimizing for cost, quality, and reliability, all through a single API endpoint.

Does OrcaRouter add latency to my requests?

The prompt grading and routing decision adds less than 1 millisecond of overhead. The total added latency is typically under 50ms, which is often offset by faster response times from better-routed models and the benefits of failover and caching.

Can I keep using my existing code with the OpenAI SDK?

Yes, absolutely. OrcaRouter is fully compatible with the OpenAI SDK. You only need to change the base_url and API key. All your existing code for chat completions, embeddings, and streaming will continue to work without modification.

What happens if a model provider (like OpenAI) goes down?

OrcaRouter's automatic failover system detects the failure and instantly retries the request against a pre-defined healthy fallback model from another provider before the original request would time out. This ensures high availability for your end-users.

Is my data used for training?

No. OrcaRouter operates as a secure gateway. Your prompts and data are not used to train their routing models. They value user privacy and data is processed only to facilitate the API request and routing logic.

When should I consider the Team or Enterprise plan?

Upgrade to the Team plan if you need advanced features for collaboration (multiple team members), stricter compliance reporting (SOC 2, HIPAA), or unlimited API keys for different services. The Enterprise plan is for organizations requiring maximum uptime guarantees (SLA), private deployment within their own infrastructure, and dedicated support.