Xentir Gemini 1.5 Pro (Sept 2024) vs Phi-4
Gemini 1.5 Pro (Sept 2024) · Google DeepMind, released 2024-09-24
Phi-4 OPEN WEIGHTS · Microsoft Research, released 2024-12-12
“Epoch AI, 'AI Benchmarking Hub'. Published online at epoch.ai. Retrieved from 'https://epoch.ai/benchmarks' [online resource].” · epoch.ai/benchmarks · Licensed CC BY 4.0 · data last verified 2026-09-09 · Methodology · Data Sources
Gemini 1.5 Pro (Sept 2024) leads on more of the 4 benchmarks where both are measured — Gemini 1.5 Pro (Sept 2024) on 4. The remaining 5 of 9 tracked benchmarks cannot be compared: 1 has a result for only one of the two models, 4 for neither. Nothing on this page is estimated to fill those in.
Head-to-head
Measured on 4 of 9 tracked benchmarks for both models — Gemini 1.5 Pro (Sept 2024) leads on 4.
| Benchmark | Gemini 1.5 Pro (Sept 2024) | Phi-4 | Leads |
|---|---|---|---|
| Epoch Capabilities Index (ECI) | 131.7 | 130.4 | Gemini 1.5 Pro (Sept 2024) |
| SWE-bench Verified | Not measured | Not measured | Not measured on both |
| GPQA Diamond | 0.572 | 0.561 | Gemini 1.5 Pro (Sept 2024) |
| MATH Level 5 | 0.704 | 0.649 | Gemini 1.5 Pro (Sept 2024) |
| OTIS Mock AIME 2024-2025 | 0.231 | 0.138 | Gemini 1.5 Pro (Sept 2024) |
| FrontierMath | Not measured | Not measured | Not measured on both |
| FrontierMath Tier 4 | Not measured | Not measured | Not measured on both |
| SimpleQA Verified | Not measured | Not measured | Not measured on both |
| Chess Puzzles | Not measured | 0.010 | Comparison unavailable — Gemini 1.5 Pro (Sept 2024) has no published result |
What remains uncertain
- Gemini 1.5 Pro (Sept 2024) has no published result on Chess Puzzles.
- 4 tracked benchmarks have no published result for either model.
A missing result means Xentir has not found one published by the source it cites, not that the model performs badly. See Data Sources.
Full profiles
Gemini 1.5 Pro (Sept 2024) → · Phi-4 → · the full AI Model Rankings table