LIVE INTELLIGENCE · part of the Xentir AI Index

Claude 3.5 Haiku (Oct 2024) vs Mistral Large 2 (2407)

Claude 3.5 Haiku (Oct 2024) · Anthropic, released 2024-10-22
Mistral Large 2 (2407) OPEN WEIGHTS · Mistral AI, released 2024-07-24

“Epoch AI, 'AI Benchmarking Hub'. Published online at epoch.ai. Retrieved from 'https://epoch.ai/benchmarks' [online resource].” · epoch.ai/benchmarks · Licensed CC BY 4.0 · data last verified 2026-07-26 · Methodology · Data Sources

Head-to-head

Measured on 4 of 9 tracked benchmarks for both models — Claude 3.5 Haiku (Oct 2024) leads on 1, Mistral Large 2 (2407) leads on 3.

Benchmark	Claude 3.5 Haiku (Oct 2024)	Mistral Large 2 (2407)	Leads
Epoch Capabilities Index (ECI)	127.5	127.5	Mistral Large 2 (2407)
SWE-bench Verified	Not measured	Not measured	Not measured on both
GPQA Diamond	0.381	0.490	Mistral Large 2 (2407)
MATH Level 5	0.464	0.448	Claude 3.5 Haiku (Oct 2024)
OTIS Mock AIME 2024-2025	0.043	0.085	Mistral Large 2 (2407)
FrontierMath	0.003	Not measured	Not measured on both
FrontierMath Tier 4	Not measured	Not measured	Not measured on both
SimpleQA Verified	0.067	Not measured	Not measured on both
Chess Puzzles	Not measured	Not measured	Not measured on both

Benchmark results measure specific tasks and should not be treated as a universal measure of model quality. A model with no row here for a benchmark has not been measured on it — that is left blank, never estimated.

Full profiles

Claude 3.5 Haiku (Oct 2024) → · Mistral Large 2 (2407) → · the full AI Model Rankings table