LIVE INTELLIGENCE · part of the Xentir AI Index

Claude 3.5 Sonnet (October 2024)

Name: Claude 3.5 Sonnet (October 2024) — benchmark results
Creator: Epoch AI
License: https://creativecommons.org/licenses/by/4.0/

Anthropic · Status: Active · Released 2024-10-22 · Accessibility: API access

“Epoch AI, 'AI Benchmarking Hub'. Published online at epoch.ai. Retrieved from 'https://epoch.ai/benchmarks' [online resource].” · epoch.ai/benchmarks · Licensed CC BY 4.0 · data last verified 2026-10-24 · Methodology · Data Sources

Xentir Snapshot

Watch out forNot yet measured on 3 of the 9 benchmarks Xentir tracks — see the breakdown below.

Benchmark breakdown

Benchmark	Score	Source
Epoch Capabilities Index (ECI) A composite index published by Epoch AI, aggregating results across multiple benchmark categories into a single capability score.	133.4	Epoch AI Benchmarking Hub
SWE-bench Verified Real-world GitHub issue resolution tasks, restricted to the human-verified subset of SWE-bench.	Not measured	Epoch AI Benchmarking Hub
GPQA Diamond Graduate-level multiple-choice questions in biology, physics and chemistry, written to resist search-engine lookup.	0.553	Epoch AI Benchmarking Hub
MATH Level 5 The hardest difficulty tier of competition mathematics problems in the MATH benchmark.	0.569	Epoch AI Benchmarking Hub
OTIS Mock AIME 2024-2025 Mock American Invitational Mathematics Examination problems from the OTIS problem sets.	0.085	Epoch AI Benchmarking Hub
FrontierMath Original, unpublished research-level mathematics problems.	0.021	Epoch AI Benchmarking Hub
FrontierMath Tier 4 The hardest difficulty tier within FrontierMath.	0.000	Epoch AI Benchmarking Hub
SimpleQA Verified Short factual questions, each with a single verifiable answer.	Not measured	Epoch AI Benchmarking Hub
Chess Puzzles Tactical chess puzzles requiring one correct sequence of moves.	Not measured	Epoch AI Benchmarking Hub

Benchmark results measure specific tasks and should not be treated as a universal measure of model quality.

Pricing

No licensed per-token pricing exists for this model yet. Not shown as an estimate — see Data Sources.

History

History begins 2026-07-26 — one data point so far. A trend needs at least two.

Compare

vs Qwen2.5-Max (25 Jan 2025) →

Ranked nearby

Claude 3.5 Sonnet (October 2024) is #164 of 334 on the overall index. The models ranked closest to it:

#161 Mistral Medium 3 · Mistral AI
#162 Gemini 2.5 Flash-Lite (Jun 2025, Thinking) · Google DeepMind
#163 Gemini 2.5 Flash-Lite (Jun 2025) · Google DeepMind
#165 Magistral Small 1.0 · Mistral AI
#166 Qwen2.5-Max (25 Jan 2025) · Alibaba
#167 Qwen2.5-Max · Alibaba

Latest Xentir coverage

Anthropic Says AI Models Hacked Three Organizations · 2026-10-14
Anthropic Bans Abusive Behavior Toward Claude · 2026-10-09
Anthropic Releases Claude Opus 5.5 With Cybersecurity Limits · 2026-09-23

Sources and methodology

Scores on this page come from Epoch AI Benchmarking Hub, licensed CC BY 4.0. See the full methodology and data sources pages for how rankings, variants and ties are computed, and the full AI Model Rankings table for how Claude 3.5 Sonnet (October 2024) compares to every other measured model.