What a large language model is
A language model assigns probabilities to sequences of text: given everything so far, how likely is each possible next token? A large language model does this with a neural network of billions or trillions of parameters, trained on an enormous corpus of text and code. Almost every current LLM is built on the transformer architecture introduced by Vaswani and colleagues in 2017, whose attention mechanism lets each token draw on every other token in the input.
Training happens in stages. Pre-training on raw text teaches the model language and a great deal of world knowledge. Further training on instructions, demonstrations and human or AI feedback turns a text predictor into an assistant that follows requests, refuses some, uses tools and writes in a consistent voice. The result is a model that can answer questions, summarize, translate, write code and plan — all by predicting one token after another.
How an LLM produces an answer
Running a trained model is called inference. The model first reads the whole input — system prompt, conversation, documents — in one parallel pass, then generates the output one token at a time, each new token conditioned on everything before it. That asymmetry is why APIs price input and output separately, with output usually several times dearer.
An LLM has no memory between requests. Everything it should consider has to be sent again in each call and fit inside its context window. Reasoning models add a hidden step: they generate reasoning tokens before answering, which improves hard tasks and bills as output.
How LLMs are sold
There are three ways to pay for an LLM:
- Per-token API access. The vendor runs the model and bills every million tokens in and out. This is what our price index tracks: the panel below counts the current public models in it, the vendors behind them and the spread of rates.
- Subscriptions. A flat monthly price for a chat app or a coding tool, with usage limits instead of a per-token bill.
- Open weights. Some vendors publish the trained parameters, so you can run the model on your own hardware or rent it from a host — see open-weight model.
The same model is often available all three ways, at prices that are not linked to one another. Which is cheapest depends on volume: our subscription vs API calculator finds the break-even point.
In our indexThe models in our index
Counted at build from our price index: every current public model with a published per-token price among the 167 current public models we price, from 33 vendors. Each rate comes from the vendor's own price page, read between 2026-07-11 and 2026-10-09. How we collect prices.
What drives the cost of using one
Once a model is chosen, the bill is tokens × rates, and four things decide the tokens:
- Prompt size — system prompt, history, documents and tool definitions, resent on every call; caching cuts the price of the repeated part.
- Answer length — billed at the higher output rate.
- Thinking — reasoning models add hidden output; reasoning effort controls how much.
- Calls per task — an agent may call the model dozens of times for one job.
Model choice moves the rate by orders of magnitude: the cheapest and the most expensive current models in our price index are worlds apart per token. Routing easy requests to a small model and hard ones to a large model is the standard way to use that spread.
Worked exampleOne assistant, a month of traffic
100,000 requests a month on GPT-6 Sol, priced at its current list rates, checked on 2026-09-30.
A hundred thousand requests a month, each a 2,500-token prompt and a 600-token answer. The same model, the same answers — the second column only changes how much of the repeated prompt is billed at the cached rate.
Change the workload or the model in the LLM API cost calculator.
Choosing an LLM for a job
Leaderboards rank models on someone else's tasks. The cheaper and more reliable route is to run a sample of your own requests through two or three candidates and compare the answers, then check four things before committing:
- Hard limits. The context window and output limit must fit your largest real request; the model must accept your input types — see multimodal model.
- Cost per task, not per token. Tokenizers differ, reasoning models add hidden output, and cache and batch discounts vary by vendor; price the workload in the cost calculator rather than comparing list rates.
- Terms. Free tiers can use your data under different rules, and Anthropic and OpenAI both add a 10% uplift for regional or US-only data-residency processing on newer models.
- Lifespan. Check whether the model already has a retirement date on the vendor's deprecation page — see model deprecation.
Whatever you pick, keep the model name in configuration. Vendors replace their lineups every few months, and the switch should be one line, not a project.
What an LLM is not
An LLM is not a database: it reproduces patterns from training and can state false things fluently, which is why products ground it with search or retrieval. It does not learn from your conversations by default — knowledge after its training cutoff arrives only through what you put in the prompt. And it does not act on its own; tool calls and agents are software around the model that executes what it asks for.