How we work

Six rules govern everything we publish. They are locked into the methodology — changing them requires a quarterly review with a written rationale, not an editorial decision. The rules below are the surface; the rest of this document is how they're operationalized.

Pricing for 291 LLM models across 22 providers is verified manually against each vendor's own canonical pricing page. No scrapers. No automated billing-API pulls. No third-party aggregators. The trade-off is honest: slower updates than a scraping pipeline would deliver, but every number has a human who checked the source URL on the date stamped next to the price.

The same model is usually sold through several doors: the vendor's own console, a cloud marketplace, a consumer subscription app, sometimes open weights you can host yourself. Each provider page lists those routes because the door you walk through changes what you pay — and the rate we publish belongs to exactly one of them.

The published per-token rate is always the first-party rate, read from the vendor's own pricing page as described above. Where a route bills differently — a marketplace markup, per-seat licensing, a region premium — the route's card says so rather than silently reusing the direct number.

Resale prices are never a source. A gateway can name its own margin and change it without the vendor doing anything, so a number read off a reseller tells you about the reseller. We'll name those routes when they matter — MIT-licensed weights served from US/EU regions solve a real data-residency problem — but the price stays theirs, not the vendor's.

Route availability is also dated and specific: which regions a marketplace serves, which model versions it actually carries, whether data stays in-region. Where we can't verify a route's terms first-hand, the card reports what the vendor documents and doesn't dress it up as tested.

We do not run benchmarks ourselves. Every quality score on this site is read off a third-party leaderboard that evaluates models under one standardized harness, and each score carries the leaderboard it came from and the date we read it. We state that plainly because the alternative is worse: a desk this size claiming its own 500-task agent runs would be claiming a compute budget it does not have, and you would have no way to check.

We lean on neutral leaderboards rather than vendor numbers because vendor-published scores are not comparable to each other. Custom scaffolding, tuned prompts and larger context budgets are all normal practice in a launch post, and none of them are labeled. One third-party harness applied to every model is a weaker claim than a self-run — but it is an honest one, and it is the same claim for every model on the board.

Every cell records its value, source, source URL, the configuration the leaderboard used, a provenance tag, and a verification date. A score whose verification date is older than 30 days is re-pulled before it can be published again; it is never quietly re-served as current. All 51 cells on the board today are neutral third-party results.

We track 8 benchmarks, and they are not all equal. 6 are core: they carry a fixed weight and are the only ones that feed a ranking. The rest are profile-only badges — shown on a model where we have them, never folded into a score. Keeping that line visible matters, because a benchmark quietly promoted from badge to ranking input would change every board on the site without anyone announcing it.

4 of the 6 core tests are flagged as saturated — 2 where the field as a whole has topped out and 2 where only the leaders have converged. We still show them, because a saturated test is useful evidence that a cheap model has caught up with an expensive one. We just don't let a one-point lead on a saturated test decide a ranking, since at that end of the scale the gap between two models is noise rather than signal.

We do not publish a page per benchmark, and there is no reproduction script to link to — see § 04 for why. What each score does carry is the leaderboard it came from, the configuration that leaderboard ran, and the date we read it, printed next to the number wherever it appears.

Editorial rankings (Top 10 lists, Best-of picks) follow a strict process: rubric locked → tools scored independently → outliers re-reviewed → final ranking computed. Sponsorship cannot move a tool up or down. Affiliate relationships exist but are disclosed per-link.

A comparison page — there are 48 of them carries no numbers of its own. Each side declares a reference into the pricing snapshot, and every price, context window and effective-cost figure resolves from that snapshot when the site is built. This is deliberate: a comparison that stores its own copy of a price will eventually contradict the model page it links to, and the reader has no way to tell which one is stale.

Benchmark bars come from third-party leaderboards, joined by the same model reference, with the source and verification date printed under every chart. When a neutral score is missing for either side, the build refuses the card and the author has to cite scores explicitly instead. That rule exists because an absent score is not a low score — closed models are routinely under-covered, and quietly rendering a gap as a short bar would read as a finding when it's a coverage hole.

The verdict table on top is the editorial layer, and it's marked as such. Category calls are a judgement, but each one carries a one-line margin naming the number and the date behind it, so you can disagree with the call while still seeing what it was based on. Where two plans of the same product are compared, benchmarks are dropped entirely: identical underlying models would render as two identical bars and imply a measurement that never happened.

Reader quotes, where a page has them, are real threads only — each with a live link, a date, and a length cap. They're context on how a tool feels in practice, never a substitute for a measurement.

We do not publish localized pricing yet. Every rate on the site today is the vendor's list price in USD, shown as the vendor publishes it. No currency conversion, no VAT or GST, no per-country availability — because none of those are things we can currently verify to the standard the rest of this document describes.

This section exists to state the plan and the gate, not to describe something already shipped. When country pages do land, the rule is that a converted price is a derived number and has to be labeled as one: the vendor's USD list price stays visible next to it, with the rate source and the date of conversion. A price that silently becomes €-denominated is a price the reader can no longer check against the vendor's own page.

Tax is the harder half and it is the reason this is gated rather than merely unbuilt. VAT and GST treatment of cross-border API billing depends on the buyer's registration status, not just their country, and getting it wrong produces a number that looks authoritative and is wrong in a way readers can act on. Until that logic is reviewed by someone competent to review it, no tax-inclusive figure ships — an absent country page is a far cheaper mistake than a confidently incorrect tax rate.

Every calculator uses verifiable math — no black-box estimation. The formula is printed on each calculator page, every rate comes from the same vendor-verified pricing snapshot as the rest of the site, and all numbers on a calculator page (presets, worked examples, comparison tables) are recomputed from that snapshot on every rebuild — nothing is hand-typed.

The LLM API cost calculator models the levers that actually move bills: prompt-cache hit rates with vendor cache-write premiums, batch-tier rates, and hidden reasoning tokens billed at the output rate. Default assumptions are shown in the UI, not hidden, and every input can be tweaked.

Token counting is labeled with one of three accuracy tiers, on every row. exact — OpenAI models counted in your browser with the o200k_base BPE, the same encoding the API uses. cal — vendors whose token counts use a chars-per-token calibration we measured on the vendor's own published tokenizer (downloaded from their official Hugging Face repos; measurement date shown on the token counter). est — vendors with no public tokenizer (Claude, Gemini, Grok, Kimi), estimated from vendor documentation. Where a closed API model is calibrated via the vendor's open-weight sibling, the page says so.

When vendors reprice or release new tokenizers, the snapshot and calibrations are re-verified and the whole section recomputes on deploy. Found math that looks wrong? Report it — calculator errors are treated as corrections, not feedback.

For "Can X do Y?" pages (e.g., "Can Claude Code work offline?"), we run the test ourselves, capture the actual output, and label answers with verification status. The tested version, date, and exact command/prompt used are shown on every capability page.

Where the answer is "no," we always provide workarounds — the alternative path or tool that does support the capability. We don't just confirm a limitation and walk away. The capability question template's "Alternative path" section is required for any "no" answer.

For every install guide, the author installs the tool on a clean machine on each platform we cover (macOS, Linux, Windows). Steps are recorded as the install runs; expected output is captured from actual terminal output, not described from memory. Tested-with version is logged in the byline.

Guides are re-verified quarterly or whenever a tool ships a new major version. Common errors come from our own install logs plus support-channel monitoring on each tool's Discord/forum/Stack Overflow tag — we add troubleshooting items as we encounter or hear about them.

Sponsored slots exist on some pages. They are visually quarantined (warm-amber dashed border, "ⓘ SPONSORED" tag, never inside editorial ranking blocks) and cannot affect editorial scoring. Sponsorship is sold at fixed published rates.

Affiliate relationships are separate from sponsorship. Many tools we recommend have affiliate programs we participate in. Each affiliate link is marked with the ↗ symbol. Affiliate revenue funds our testing budget. Affiliate status cannot move a tool up or down in editorial rankings — and we cite tools without affiliate programs (DALL-E, Imagen, OpenAI direct API) on their merits when they win on rubric.

When we get something wrong, we log it. The corrections page lists every correction with: date issued, page affected, what was wrong, what's now correct, who flagged it. We don't silently edit. We don't claim things were always correct that were not.

Severity levels: minor (typo, broken link) → fixed silently with timestamp on page. Material (incorrect price, wrong benchmark score, wrong feature claim) → corrected page + entry in /corrections/ + email reply to whoever flagged it. Severe (incorrect ranking, retracted claim) → corrected page + corrections log + apology paragraph in next quarterly editorial review.

The canonical record of every correction we've issued lives at /corrections/ — each entry includes the original claim, the corrected claim, the date, and the page affected.

Founder signs off on every published claim. AI assists with drafting and formatting; the human author makes every editorial decision and verifies every fact. As the desk scales, additional named reviewers will appear here with verifiable handles — second-pair-of-eyes hires are welcome.

Every change to this methodology is logged. The page you're reading is v1.3. The log below starts where the document does — this site published its first page in May 2026, and there is no earlier methodology to point at.