How routing works
The simplest routers are rules: classification requests to a small model, code generation to a coding model, anything over a length threshold to a model with a large window. Learned routers go further and predict, per request, whether a cheaper model's answer will be good enough. The RouteLLM paper (Ong and colleagues, 2024) trained such routers on human preference data and reported cutting costs by over two times in certain cases without a loss in response quality on its benchmarks.
A cascade is the sequential version: try the cheap model first, check the answer — with a validator, a confidence score or a test — and escalate to a stronger model only when the check fails. Cascades suit tasks where answers can be verified automatically, such as code that must pass tests or extractions that must match a schema.
Why routing saves money
Rates differ enormously between the smallest and largest current models in our price index, while the share of requests that genuinely need a frontier model is usually small. Routing turns a product's cost from the price of its best model into a blend weighted by how traffic splits. The worked example below prices the same month of traffic on one large model and on a split.
Routing also happens inside one model. Reasoning models let you set effort per request — low for a quick reply, high for a hard problem — which captures some of the same saving without operating two models.
Worked exampleOne large model vs a routed split
100,000 requests a month on Claude Sonnet 5.5, priced at its current list rates, checked on 2026-10-09.
A simplification for illustration: the second column scales the Sonnet traffic down to the 20% of requests a router would send it. The other 80% would run on a small model — price that share on its own model page, such as Claude Haiku 5.5, and add it to see the full routed bill.
Change the workload or the model in the LLM API cost calculator.
Where routing goes wrong
- Misrouting hard requests — a cheap model's wrong answer can cost a retry on the expensive model, or a user, which outweighs the saving.
- A costly router — a classifier that calls a large model on every request eats into what it saves.
- Lost cache hits — splitting traffic across models splits the prompt cache; each model warms its own.
- Drift — model releases and retirements change what each tier can do; routes need re-evaluation.