AI Index Methodology

The Xentir AI Index is compiled automatically from published third-party benchmark results. Xentir does not run these benchmarks. This page documents the actual rules the sync and ranking code apply — nothing here is aspirational.

Where the numbers come from

The current data source is the Epoch AI Benchmarking Hub: eight benchmarks Epoch itself runs — GPQA Diamond, MATH Level 5, SWE-bench Verified, OTIS Mock AIME 2024–2025, FrontierMath, FrontierMath Tier 4, SimpleQA Verified, Chess Puzzles — plus Epoch's own composite, the Epoch Capabilities Index (ECI). See Data Sources for the full register and why every other candidate source is currently not used.

“Epoch AI, 'AI Benchmarking Hub'. Published online at epoch.ai. Retrieved from 'https://epoch.ai/benchmarks' [online resource].” · epoch.ai/benchmarks · Licensed CC BY 4.0

Identifying a model (alias normalisation)

Epoch's own "Model version" field carries a reasoning-effort or thinking-budget suffix on the same model — e.g. claude-opus-5_max, gpt-5.5_xhigh, gpt-5.5_128k. These are the same model measured under different settings, not different models. The suffix is recognised against a fixed word list (effort words: max, high, medium, low, minimal, xhigh, none, thinking, nothink, and others) or a token-budget pattern like 32k/128k. What remains is the canonical model identity; the suffix is stored separately as the variant. Every raw string Epoch used is kept in an aliases table, so the normalisation is auditable back to source.

The variant rule — what gets ranked, and what gets printed

A model is ranked on its best-scoring variant for a given benchmark. The variant is printed beside the score whenever it matters — e.g. "0.835 (max)" — because a reader who wants that exact number needs to know what to run to reproduce it.

The one exception: some benchmarks (the ECI composite, and occasionally others) carry the identical value across every variant a model has — Epoch repeats one composite number across every effort-tier row rather than computing it per tier. Printing an arbitrary variant label next to a number that every variant shares would imply a choice was made where none was; there, and only there, the number is printed bare, with no variant. A model measured under only one variant is not this case — there is nothing to have tied, so its one real variant is still shown.

Multiple runs of the same model and variant

Epoch occasionally publishes more than one independent evaluation run for the same model at the same variant on the same benchmark (confirmed on SWE-bench Verified: two genuine re-runs each, with distinct run identifiers and start times). These are measurement noise around one underlying value, not different facts, so they are averaged — the same reasoning already applied to averaging multiple scorers within one run. The stored score keeps the source run identifiers that were averaged, so a blended number is always traceable back to the runs behind it.

Release dates

A model holds at most one "released" event. If Epoch's own source disagrees with itself on the date (this happens — one model's two variant rows carried release dates two days apart), the earliest is kept as the model's release date and the disagreement is recorded, not silently dropped or left to duplicate.

Benchmark versioning

Every score is stamped with a source_version derived from the exact downloaded file (its SHA-256, truncated). If Epoch changes how a benchmark is measured, that becomes a new version, not a silent update to old numbers — scores from different versions are never averaged together or plotted on one line without saying so. Every score also carries its higher_is_better flag and a verified_at timestamp, both shown on the page.

What a composite score does and does not mean

The Epoch Capabilities Index is Epoch's composite, not Xentir's. It is never renamed, never rescaled, and never presented as though Xentir computed it. Xentir's "Overall" ranking tab uses ECI as published, with its citation visible on the page.

Source precedence

Only one source (Epoch AI) currently feeds the ranked benchmarks. When a second source is added, this section will state, before it ships, which source wins when two disagree on the same model and benchmark — that rule does not exist yet because there is nothing yet for it to arbitrate.

Pricing coverage

Reference prices on the leaderboard and the Best Value card are read directly from OpenAI, Anthropic and Google's own published per-token price pages, re-checked weekly, and shown as "price as published by {vendor}, checked {date}" — never estimated, interpolated or carried forward silently from an old read. Not every model has a price cell: a model has to still be listed on its vendor's current pricing page, and several vendor entries publish a context-length-tiered or per-modality price in one cell rather than one flat per-token figure — those are left blank rather than guessing which figure is "the" price. See Data Sources for per-vendor coverage detail.

What is not built yet

Inference speed (Fastest) is not shown as an estimate anywhere on the Index — Xentir has no licensed or self-measured speed data at all, and it renders as an honest empty state rather than a guess.

Benchmark results measure specific tasks and should not be treated as a universal measure of model quality.