Strong correlation. A rough forecast, not a substitute for testing.
Across 14 screened model pairs, Pearson r is 0.892, Spearman ρ is 0.894, and the linear fit explains 79.5% of the variation in this sample. Leave-one-model-out prediction error is 10.9 percentage points RMSE. These are calculations from the linked source snapshots, not numbers reported by either benchmark.
The two models you asked about
GLM 5.3 (max): 29.27% is already measured on Epoch’s Tier 4 v2, with a 7.19-percentage-point standard error. The evaluation started on August 25, 2026; run ID 4S3ZrJDkwh4gmGrmJQdvzR. No GLM 5.3 Apex result appears in the downloaded MathArena leaderboard or the checked model cards. It therefore is not assigned an invented x-coordinate on the scatter plot. GLM 5.3 has publicly available, license-restricted weights; Epoch’s accessibility metadata still says API access. This note does not update the earlier chart.
DeepSeek V4.1 Flash: a conditional forecast of 49.2%, with a nominal 95% prediction interval of 26.3–72.1%, if its manufacturer-reported 65.6% Apex result is comparable to the independent Apex scores. There is no V4.1 Flash result in the downloaded Epoch data. This is the new September 10 model, not July’s V4 Flash.
How the fit works
Ordinary least squares, one equally weighted row per matched model: Tier 4 (%) = 8.09 + 0.627 × Apex (%). We fit only independent MathArena Apex observations against Epoch’s exact FrontierMath-Tier-4-v2-Private task; never Apex Shortlist, MathArena’s overall score, earlier Tier 4 versions, or vendor scores. Reasoning effort and endpoint metadata were inspected manually. Benchmarks still differ in prompts, tools, output budgets and test dates, so these are screened model matches, not identical evaluation protocols.
The shaded band and orange whisker are 95% prediction intervals for one additional model, not a confidence interval on the mean fit. Formula: fitted value ± tn−2,0.975 × residual SD × √(1 + 1/n + (x−x̄)²/Σ(x−x̄)²). Displayed bounds are clipped to 0–100%. We do not separately add benchmark sampling error: the regression residual already includes score noise. The interval treats the Apex input as fixed and excludes any extra uncertainty from vendor reporting, changing endpoints, model-family dependence, or data contamination.
Omitting each provider in turn moves the 65.6%-Apex point prediction from 42.0% to 52.5%. Adding the ambiguous DeepSeek April-preview match yields r = 0.862 and a 48.2% forecast. These are sensitivity checks, not independent validation. Models from the same lab are related; fourteen rows do not represent fourteen independent experimental replications.
MathArena marks Apex deprecated. It contains just 12 public problems, so many runs do not create many independent problems. Post-release contamination warnings are not evidence of actual contamination, but are a reason to be cautious. Its public final-answer format is different from Epoch’s private, research-level Tier 4 task.
Matched data
Apex errors are the source’s ±95% confidence half-widths; Tier 4 errors are ±1 standard error. They are different quantities. Points use the reported means.
| Model | Weights | Apex % ±95% CI | Tier 4 % ±SE | Epoch version / effort |
|---|---|---|---|---|
| Opus 4.8 | Closed | 81.25 ± 7.81 | 56.10 ± 7.85 | claude-opus-4-8_max |
| GPT-5.5 | Closed | 80.21 ± 7.97 | 72.50 ± 7.15 | gpt-5.5_xhigh |
| GPT-5.4 Pro | Closed | 69.79 ± 9.19 | 58.54 ± 7.79 | gpt-5.4-pro-2026-03-05_xhigh |
| Kimi K3 | Open | 65.62 ± 9.50 | 39.02 ± 7.71 | kimi-k3_max |
| Gemini 3.1 Pro | Closed | 60.94 ± 6.90 | 26.83 ± 7.01 | gemini-3.1-pro-preview |
| GPT-5.4 | Closed | 54.17 ± 7.05 | 49.00 ± 8.00 | gpt-5.4-2026-03-05_xhigh |
| Gemini 3.5 Flash | Closed | 32.29 ± 9.35 | 26.83 ± 7.01 | gemini-3.5-flash_high |
| Gemini 3.6 Flash | Closed | 26.04 ± 8.78 | 21.95 ± 6.54 | gemini-3.6-flash_high |
| Kimi K2.6 | Open | 23.96 ± 8.54 | 25.64 ± 7.08 | kimi-k2.6 |
| GLM 5.2 | Open | 19.79 ± 7.97 | 29.27 ± 7.19 | glm-5.2_max |
| Gemini 3 Flash | Closed | 15.62 ± 5.14 | 17.07 ± 5.95 | gemini-3-flash-preview |
| Sonnet 4.5 | Closed | 1.56 ± 1.75 | 2.44 ± 2.44 | claude-sonnet-4-5-20250929_32K |
| GPT-5 | Closed | 1.04 ± 1.44 | 21.95 ± 6.54 | gpt-5-2025-08-07_high |
| Gemini 2.5 Pro | Closed | 0.52 ± 1.02 | 0.00 ± 0.00 | gemini-2.5-pro |
Excluded and unmatched models
Six plausible matches rejected or kept only for sensitivity
- Claude-Opus-4.7 (xhigh): Apex uses xhigh effort; Epoch uses max.
- Claude-Opus-4.6 (High): Apex uses high effort; Epoch uses max.
- DeepSeek-v4-Pro (Max): MathArena uses a mutable endpoint with April 24 metadata. Preview versus August revision cannot be pinned confidently; sensitivity check only.
- DeepSeek-v4-Flash (Max): MathArena lists the April 24 release; Epoch's Flash score is July 31. Do not join revisions.
- GPT-5.2 (high): Apex uses high effort; Epoch uses xhigh.
- GPT-5-mini (high): MathArena's display says high but its endpoint says gpt-5-mini--low.
Kimi K2.6 is matched by the endpoint and release identity; its MathArena auxiliary Hugging Face ID still says K2.5, which appears inconsistent with those fields. Google’s named high settings and documented defaults are retained where Epoch does not encode effort in the ID. One-day release-date differences are not treated as model revisions. Many newer models have no Apex result because this old benchmark is no longer the main evaluation target.
Sources and reproducibility
- MathArena Apex leaderboard and its table endpoint; individual model/configuration pages are linked in the table.
- Epoch benchmark CSV and Tier 4 v2 chart.
- DeepSeek V4.1 model card for the vendor-reported Apex input and same-table comparisons.
- GLM 5.3 model card and Apex methodology.
Snapshot: September 10, 2026. Raw inputs, exact model mapping, exclusions, scripts and checks are retained in the workspace. The downloadable CSV is the fitted dataset; JSON includes regression and sensitivity results.