Open Weight vs Frontier Models: The Cheapest Option Is Often Neither

“Do I really need to pay for a frontier model, or is open weight good enough now?” Most answers rest on one leaderboard or a price table. So we lined up nine public tests that record cost per task, plus Epoch AI’s capability index, human votes and real usage, and asked what a given score costs. A setting, below, means one model at one effort level.

An open-weight model is one whose weights anyone can download and run, whatever the licence. A frontier model here is a closed-weight model within 5 points of today’s best closed score on Epoch’s Capabilities Index: eight models, including GPT-6 Astra, Claude Opus 5.5 and Gemini 4 Argon. GPT-6 Sol misses that line by 0.28. Every other closed model we call other closed: small ones like GPT-6 Luna and Claude Haiku 5.5, and last generation’s like GPT-5.6 Terra. That third group turns out to matter most, so the choice is not two-way.

Share of 478 test scores where the cheapest setting at that score or better was a closed model outside the frontier

52%

Source: Epoch AI archive and OpenRouter, 2 Nov 2026

Closed settings beaten on cost by an open setting, of those an open setting can match. Open reaches 186 of 280

67%

Source: Epoch AI archive, list-price tests

How far the best open-weight model trails today. It averaged 1.6 in late 2024 and 5.6 so far in 2026, and it moves in a sawtooth

6.3 months

Source: Epoch AI Capabilities Index, our calculation

What we analysed for open weight vs frontier models

The scores come from one archive, the real spend and usage from another, and neither rests on a single leaderboard’s method:

1. Epoch AI benchmark archive. Epoch AI, CC BY, 2 November 2026. Benchmark results for hundreds of models, the published cost per task on seven tests, and the Capabilities Index with its bootstrap samples. CursorBench reaches us through it, and we checked two of its rows against Cursor's page. Open the original.

   benchmark scores

   2,944

   tests

   27

   models

   292

   scores with a cost

   736

2. OpenRouter public datasets. OpenRouter, CC BY, 2 November 2026. Real spend per task on each host for two tests, 30 days of token share, and its model list. Open the original.

   results by model and host

   2,245

   token share rows

   1,530

   models listed

   451

3. Arena leaderboard dataset. Arena, on Hugging Face, CC BY, 28 October 2026. Human preference votes by category, and the Agent Arena board. Open the original.

   text board rows

   11,132

   agent board rows

   57

4. Official list prices, Kaggle series. Transcribed weekly from vendors' pricing pages, 1 November 2026. What each lab charges per million tokens, including DeepSeek's peak and off-peak rates. Open the original.

   price rows

   120

5. 

   Analysed in Infograph. Twelve tables, queried in one place, so a score could sit beside its price, its host and its usage. Fourteen open-weight models were checked against Hugging Face to confirm their weights can be downloaded. Open the interactive report.

What is the cheapest way to reach a score

Pick any score a tested setting reached, released since April and scoring at least 10%, and find the cheapest setting from any group that does as well or better. Across all nine tests that is a closed model outside the frontier at 249 of 478 scores, open weight at 154 and a frontier model at 75. Those closed models are not all small: Claude Haiku 5.5 and GPT-6 Luna take many, and so do last generation’s GPT-5.6 Terra and Sol.

The split depends on the test. On the seven tests that publish costs at list price, those closed models hold 64%, open weight 22% and frontier 14%. On OpenRouter’s two tests, which record real spend at a median host, open weight holds 58%. Nearly all of that is the airline agent task, at 57 of 59. Graduate science questions split almost evenly.

Open report

Head to head, open weight often undercuts a frontier model: 76 of 130 comparisons at the same score or better. That leans on two tests, WeirdML at 19 of 20 and the airline task at 33 of 35. Without them it is 24 of 75, and maths proofs, long coding tasks and science questions go the other way. Beating every closed setting at once is rarer: 41 of 140, and the airline task is the only test where that happens more often than not.

Open report

Why a low token price doesn’t settle it

Take one pair on Cursor’s coding test. DeepSeek V4.1 Flash 1017 at maximum effort scores 47.7% and Claude Sonnet 5.5 at high effort 47.8%. DeepSeek lists at $0.15 in and $0.60 out per million tokens off-peak, double at peak. Sonnet lists at $2 and $10. So DeepSeek is 7 to 17 times cheaper per token, yet the task costs $0.98 against $1.20. DeepSeek still wins, by 18%, but the gap has nearly gone. It takes 187 steps to Sonnet’s 41, and Cursor counts 537,047 tokens against 37,391.

Those counts cannot be the whole bill: at the top output rates they come to at most $0.64 and $0.37, so much of each cost lies elsewhere, most likely in input and cache. Cursor doesn’t define the count or say which DeepSeek rate it applied, so we can show the bills without fully explaining them. The direction holds on OpenRouter, at a smaller size: among models released since April, open-weight ones wrote a median 28,844 output tokens per science question against 8,009 for closed, and 13,026 against 7,236 per airline task.

The token advantage varies too: Moonshot lists Kimi K3 at $3 and $15, above Sonnet. And the effort dial can move a score by up to 20 points on Cursor’s test. Claude Opus 5.5 scores 43.7% for $1.17 at low effort and 57.8% for $13.43 at maximum, 11.5 times the money for 14 points. Costs here are per task attempted; on this test even the best model passes under 60%.

Open report

Where open weight wins most

The airline test is the case against our headline, and it is real money on a real kind of work: an agent following a policy and calling tools. Most models tested land between 68% and 80%, so price decides. GLM-5.3-Flash scores 74.8% for $0.013 per task at a median host. Claude Sonnet 5.5 scores 78.0% for $0.17, about 14 times as much, for 3 more points. MiMo-V2.6-Flash beats Claude Haiku 5.5 on both score and cost, and GLM-5.3-Flash matches Haiku on cost with a higher score.

If your work is routine, with a clear right answer, look at this result first. Machine learning code on WeirdML also goes open weight’s way, with 19 of 20 settings beating frontier and DeepSeek’s Flash family holding the cheap end.

Open report

How far open weight trails the frontier in months

The best closed model on Epoch’s index scores 169.06 and the best open-weight model 158.83. Using Epoch’s published method on today’s scores, open weight trails by 6.3 months. It is a sawtooth, so one date proves little. In the summer of 2024 Llama 3.1-405B was not clearly behind, though about three months back on point estimates. The lag then averaged 1.6 months in the second half of 2024, 2.8 and 4.4 in the two halves of 2025, 6.2 in the first half of 2026 and 4.8 since July, after Kimi K3 cut it to 3.3. Each big open release closes it for a while and the next closed one reopens it. There is no sign of it closing for good, and two years is too short to call a trend.

Epoch reported four months for early 2026, or six on point estimates. The same method on today’s revised scores gives us 6.2 and 7.0 for that window, and we cannot fully explain the difference. Treat these figures as less settled than they look.

Open report

Why one leaderboard can show the gap closed

The newer the test, the wider the gap looks. On tests from before 2026, the best open-weight score is a median 88% of the best closed score. On tests from 2026, it is 69%. Most old tests are near their ceilings: on OTIS Mock AIME, open weight reaches 99.4% and closed 100%. On FrontierMath Tier 4 v2, it is 46.3% against 100%. Both are true.

By task, long agent runs show the widest gap, at a median 61%, against about 82% for coding, maths and reasoning. Writing has no current objective test. Arena’s voters are the only recent view, and they put open weight 70 rating points behind on creative writing, against 36 on coding.

Open report

The same open-weight model is not always the same product

Anyone can serve open weights, and the score you read may not describe the host you get. On OpenRouter’s science test, 10.6% of open-weight hosts scored more than 5 points below that model’s median host, against 1.3% for closed. On the airline test it was 5.9% against none. Among models with six hosts or fewer, it is 7.9% against 1.3%, and 11.5% against none. Most hosts agree: DeepSeek V4.1 Flash runs from 84.0% to 90.9% on 38 of its 39 hosts, with one at 72.3%. Prices vary more, from $0.007 to $0.050 per question. Taking each model’s typical gap between two hosts, the median model’s is 2.6 points for open weight and 0.4 for closed, and some of that is sampling noise. We can’t tell from this data what causes a low score.

Open report

Where AI benchmarks disagree about open weight

The same week gives different answers, because each measures something else. Artificial Analysis rebuilt its index on 27 October and its open-weight gap went from 3 points to 12, with about 60% of it now on private tests. Even that 12 uses Kimi K3.1 Pro, whose weights it notes are not yet released; models with weights out score 32, which is 26 behind the leader’s 58. Stanford’s AI Index put the lead at 3.3% on Arena in March. On Arena today MiMo-V2.6-Pro ranks 12th overall and 4th on coding, while on the Agent Arena board it ranks 17th.

Usage points the other way: open-weight models took 67.4% of OpenRouter’s tokens from 3 October to 1 November, against 20.6% for closed, with 11.9% long tail or unclassified and 6.1% of all tokens on free-tier open models. That is one router’s developer traffic, and tokens are not tasks: if open models use more per task, their share of work is smaller.

These sources are not independent of each other or of the labs. Epoch says its index draws on over 50 benchmarks, so its parts are not separate evidence. Nathan Lambert notes that Artificial Analysis and Arena earn revenue from labs they evaluate.

We would trust composites for direction, tests of real work for what a model can do, and your own runs for what it costs. That last part is a data problem: this answer only appeared once 2,944 scores, per-host spend and list prices sat in one place, and the same is true of cost per job in your own business, where the bill and the outcome usually live in different systems.

What could make this wrong

The cheapest rival is not the typical rival. Our counts take the single cheapest setting at each score, the best of many. Flip it: of 280 closed settings released since April on the list-price tests and scoring 10% or more, an open setting reached the score of 186, and it was cheaper for 125 of those, 67%. A closed model picked without care is usually beaten. The other 94 have no open match at any price.

Two labs do much of the work. Of the 221 list-price scores held by closed models outside the frontier, OpenAI holds 123 and Anthropic 73. Without Anthropic those models still lead open weight, 118 to 101 of 250 scores. Without OpenAI as well, open weight leads, 105 of 151. So “often neither” rests heavily on those two labs and on their prices staying where they are.

Where you draw the frontier line. At 3 points instead of 5, closed models outside the frontier hold 223 of the 345 list-price scores; at 8 points, 183, still the most.

These are list prices. None of these tests measures running weights on your own hardware, a main reason to want them, and at large, steady volume that could change the sums. We cannot tell whether any closed price covers its cost. Prices also move fast on both sides: GPT-5.4 scored 157.14 at $2.50 and $15 in March, and seven months later Claude Haiku 5.5 scores 160.37 at $0.10 and $0.50, and DeepSeek V4.1 Flash 1017 scores 158.83 at $0.15 and $0.60 off-peak.

Benchmarks are not your work. Cursor sells its own models and warns that small differences “may not be statistically meaningful”. Coverage is thin: 10 open settings on CursorBench and 3 on FrontierSWE, and frontier models have not run on the two ARC tests. Only 9 tests here carry costs, and Epoch notes open models tend to do worse on private tests.

And none of it measures why many choose open weights: keeping data on your own machines, fine-tuning, no forced retirements, no single vendor. If those matter to you, these scores don’t decide it.

This report describes public data as of 2 November 2026. It does not forecast where models or prices will go, and it is not purchasing or investment advice.