How many tokens
does $1M buy?
API pricing and benchmark-based self-hosting, side by side. Each row spends the entire budget on that model—not a share across all six. Read the vLLM / SGLang audit ↓
GPU cost assumptions ·
$7/B200-hour is rounded from Lambda; $8/B300-hour is near Nebius’s $7.85 GPU-hour listing. $10/GB300-hour is an explicit scenario assumption, not a live quote: Together lists GB300 as contact-sales. Extra charges, capacity minimums and negotiated rates can change costs.
All quantities are generated output tokens. B = billion; T = trillion. The two shaded columns default to eight uncached input tokens per output token, matching the benchmark ratios. The output-only column is an upper bound that ignores API input charges.
| Model / API provider | API $ / 1M input / output | API output-only upper bound | API tokens 8:1 input:output | Self-hosted tokens 8:1 input:output | GPU setup / quantization reported serving benchmark |
|---|
Conditional, not guaranteed: Qwen Max uses forced speculative acceptance (3.3); Qwen 27B is a third-party vLLM run. GPU budgets assume the rates above and exclude operations costs; $10/GB300-hour is hypothetical. Flash has no verified matching serving benchmark, so no GPU estimate is shown. Evidence and caveats ↓
GPU output tokens = budget × 3,600 × output tok/s per billed GPU × productive fraction / GPU-hour price
What the $1M baseline means
In the output-only baseline at off-peak API prices, DeepSeek V4.1 Flash buys 1.67 trillion output tokens, versus 66.7 billion for Kimi K3—25× as many tokens per dollar. This is a quantity comparison, not a claim of equal answer quality, useful training data, or throughput.
With one uncached input token per output token, the same budget produces approximately 125B Qwen Max tokens, 286B Qwen 27B tokens, 55.6B Kimi tokens, 172B GLM tokens, 379B DeepSeek Pro tokens, or 1.33T DeepSeek Flash tokens.
Assumptions that matter
- Qwen uses Alibaba’s Singapore / International list prices. Regions and resellers differ. The official Max global-region output rate is $4.951/1M, implying about 202B tokens per $1M. The 27B Beijing rate is $1.696/1M, implying about 590B. Those are alternate regional offers, not the default scenario.
- DeepSeek uses off-peak prices by default. Peak rates are double, so the output-only quantities halve: Pro about 253B, Flash about 833B. Peak windows are 01:00–04:00 and 06:00–10:00 UTC on weekdays; all other times are off-peak. Rate limits and available capacity still apply.
- “Flash” means today’s V4.1 Flash. Pro means V4 Pro 0813. The official Pro endpoint is scheduled to route to V4.1 Flash from September 14, 2026, 12:00 Beijing time. Its row is a price-snapshot calculation—not a promise that $1M can be spent generating Pro tokens before retirement.
- Generated tokens include billed reasoning tokens where applicable, not just the final answer. Tokenizers also differ; equal token counts are not equal text or capability.
- The output-only API column excludes input charges; the selected-ratio API column includes uncached input. Both exclude tool, tax, payment and other charges. No batch, promotional or negotiated volume discounts. This is not a cheapest-provider survey. At $1M, a provider may offer a custom quote; no such discount is assumed.
For synthetic-data generation, only a fraction of these tokens may become accepted examples. Filtering, verification, failed attempts and repeated context all reduce the useful yield.
Reported throughput, with the units corrected.
The earlier GPU ranges were assumptions, not validated results. This section replaces them as the evidence-backed reference. It uses reported serving measurements for the exact checkpoint, including quantized versions where available. We did not run these benchmarks ourselves.
The largest accounting trap: most SGLang results count input + output tokens. On an 8,192-input / 1,024-output workload, divide total throughput by 9 to obtain generated output throughput. For Qwen Max, also charge for 24 allocated GPUs, not only the 20 active GPUs. None of these rates should be transplanted from GB300 to B200.
These are selected throughput-oriented operating points, not a matched cross-model benchmark or a best-engine contest. All rows use 8,192 input / 1,024 output except Qwen 27B (2,048 / 256); both ratios are 8:1. The merged table defaults to the same ratio for API input charges. Changing that control updates only the API column, not the measured GPU throughput. The raw audit download retains a fixed 8:1 API comparison.
How strong is each row?
- Qwen3.8 Max-class NVFP4 — conditional speed test. SGLang reports 5,126 total tok/s per active GB300: 12 prefill + 8 decode GPUs, 24 allocated, 36 output tok/s/user. That becomes 475 output tok/s per billed GPU. The PD measurements force speculative acceptance length to 3.3; real math acceptance may differ. The FP8 endpoint reports 3,532 total tok/s/GPU on 32 prefill + 16 decode GPUs, equivalent to 392 output tok/s/GPU. These are different deployments, not an isolated quantization A/B.
- Qwen3.8 27B — third-party vLLM measurement. ThakiCloud’s B200 comparison reports BF16 2,141.4 versus mixed NVFP4 MLP + FP8 attention + FP8 KV 3,586.3 output tok/s at concurrency 128, a 1.675× improvement. Do not divide these output-only numbers by nine. At the default $7/hour, that is 1.10T versus 1.84T generated tokens per $1M. This was not a vLLM-team benchmark, and its quality check was limited to MMMU multiple-choice rather than FrontierMath.
- Kimi K3 — native MXFP4, not an extra 4-bit saving. The SGLang table reports 1,987 total tok/s/GPU on eight B300s at concurrency 64 with DSpark (mean TPOT 24.47 ms), or 221 output tok/s/GPU. Without DSpark it reports 1,395 total; the NVFP4 + DSpark variant reports 1,946, slightly below native MXFP4 here. Separately, the GB300 PD result is 2,808 total tok/s/GPU at 8K/1K, yielding 312 output tok/s/GPU, or 112B tokens/$1M at $10/hour. vLLM’s 370 tok/s headline is per-user batch-one speed, not output throughput per GPU.
- GLM 5.3 — actual 5.3 NVFP4 measurements. The SGLang cookbook data reports 8,537 total tok/s/GPU on four GB300s at concurrency 256, or 949 output tok/s/GPU. On eight B300s it reports 5,416 total, or 602 output tok/s/GPU (271B tokens/$1M at $8/hour). These are not GLM 5.2 proxy results. The FP8 B200 entry reports 5,025 total (558 output) but forces acceptance length to 2; do not treat that as a natural-output A/B against NVFP4.
- DeepSeek Pro 0813 — native FP4 measurement. For the exact
pro-officialvariant, the SGLang source reports 7,475 total tok/s/GPU on four GB300s, concurrency 512, or 831 output tok/s/GPU. This buys throughput at poor latency: mean TPOT 120.69 ms and TTFT 34.09 s. The concurrency-256 point is 6,758 total, or 751 output tok/s/GPU. The separate low-latency DSpark rows use simulated acceptance; they are not the rows used here. - DeepSeek V4.1 Flash — unverified. No matching end-to-end vLLM/SGLang serving benchmark was found for the new checkpoint. The published V4 cookbook covers older Flash, Flash 0731 and Flash Vision models, which are different checkpoints. Its reference inference example is not a production serving measurement. The earlier 0.26–1.03T range is withdrawn as a planning recommendation.
Quantized checkpoints and quality
- Qwen Max has RadixArk NVFP4 and Inferact NVFP4 / MXFP4 variants. vLLM’s release post reports GSM8K/AIME25 checks and single-node B300 serving for FP4; that does not make the GB300 PD result a single-node result.
- Qwen 27B’s measured mixed checkpoint is ThakiCloud NVFP4 + FP8 attention. The benchmark’s extra FP8 KV option is part of the speedup.
- GLM 5.3 NVFP4 quantizes routed experts while retaining attention and other modules in BF16: 465 GB versus 1,507 GB BF16. Its publisher reports GSM8K 97.42% and AIME 2026 pass@1 94.17%, near its BF16 check under the same protocol.
- NVIDIA’s DeepSeek Pro 0813 NVFP4 supports vLLM and SGLang. However, the model already ships MXFP4 experts: the conversion changes scale representation and activation handling, and the checkpoint is slightly larger, not half-sized. Its published deployment/accuracy validation does not establish a serving-throughput multiplier, so none is assumed.
- No supplied quantization check establishes unchanged FrontierMath Tier 4 performance. For synthetic math data, benchmark accepted, verified solutions per dollar as well as raw tokens.
Revised conclusion: quantized Qwen 27B is the clearest throughput-per-dollar win here. Reported large-model setups land around 0.1–0.35T output tokens per $1M at these rental assumptions. That is workload-specific, not a bound on output-heavy math generation. API comparisons improve for self-hosting when input charges are included, but caching, API batch discounts, lower utilization and ops costs can reverse the result.
Archived initial GPU assumptions (not measured benchmarks)
Self-hosting is a throughput bet.
Archived initial assumptions, superseded by the benchmark audit. These remain editable for sensitivity analysis; they are not reported measurements. The Flash row is retained only as the historical hypothetical calculation, not an endorsed estimate.
These are conditional planning estimates, not six measured benchmarks. We assume a continuously supplied batch-generation queue, roughly 1–4K input and 4–16K output tokens per request, and no strict interactive-latency requirement. The throughput ranges below are judgment calls for that workload; they are not confidence intervals. Actual performance can fall outside them.
Prices are rounded from Lambda’s current rental page: $7/B200-hour for one-node setups (listed $6.69–6.99), and $10/B200-hour for networked multi-node setups (16-GPU cluster listed $9.86). Multi-node B200 over a network is not equivalent to GB200 with multi-node NVLink. The large-model configurations below are sizing assumptions, not validated deployment recipes.
“Productive fraction” means time spent sustaining the assumed end-to-end throughput, not GPU FLOP utilization. Lower it for empty queues, restarts and maintenance. The API input-ratio control does not change the GPU throughput assumptions.
| Model / assumed configuration | Whole setup $/hour | Assumed output tok/s whole setup, not per GPU | GPU output tokens | GPU $/1M output | API output tokens |
|---|
Example: eight B200s at $7 each cost $56/hour. If the whole server sustains 4,000 generated tokens/second, $1M buys 257B tokens, or $3.89 per million. At 50% productive time, it buys 129B. Input processing is part of the assumed end-to-end throughput; it is not free GPU work.
What supports these assumptions?
- Qwen3.8 Max → Qwen3.8-2.4T-A95B: 2.4T total / 95B active parameters. The NVIDIA FP8 recipe uses 16 GB200 or GB300 GPUs and multi-node NVLink. Our 16-B200 networked configuration and 2,000–8,000 output tok/s are unmeasured extrapolations. The open checkpoint is not guaranteed to reproduce every feature or revision of the Max API.
- Qwen3.8 27B: one B200, BF16 weights, assumed 1,000–2,500 output tok/s. ThakiCloud reports 2,141 output tok/s for BF16 at concurrency 128, with 2,048 input / 256 output tokens. It reports 3,586 with mixed NVFP4/FP8, but our baseline does not assume that extra quantization or its speedup transfers to long math reasoning.
- Kimi K3: 2.8T total / 104B active, native MXFP4 experts. Sixteen B200s is a capacity-sizing assumption. The NVIDIA guide has 16-GB200 workers, but its short-output agentic trace reports only about 20–33 output tok/s/GPU. Our 1,000–6,000 output tok/s per 16-GPU group explicitly assumes a different, much more heavily batched, output-heavy workload; it is not that benchmark result.
- GLM 5.3: eight B200s with FP8 weights; 2,000–8,000 output tok/s is assumed. The NVIDIA 5.3/5.2 guide confirms FP8 and NVFP4 serving options, but its performance table was measured on GLM 5.2, not 5.3, on a different agentic workload. It does not validate this range.
- DeepSeek Pro 0813: eight B200s with native MXFP4 experts; 2,000–8,000 output tok/s is assumed. NVIDIA’s 0813 recipe uses eight GB200s or H200s and a roughly 832-GiB checkpoint. Its interactive agentic results are much lower than this output-heavy scenario. Results for the original V4 Pro are not measurements of 0813.
- DeepSeek V4.1 Flash — highest uncertainty: eight B200s and 4,000–16,000 output tok/s are speculative planning inputs. The new model has a 552B backbone plus 196B Engram parameters; sizing cannot ignore that extra memory. The official TP8 example is a reference implementation, not a production throughput benchmark. No validated end-to-end serving benchmark for this new checkpoint was found. This is not the older 284B V4 Flash.
How to use the estimate
- Measure before committing. Replace each throughput range with completed output tokens divided by total wall time on your actual prompts. Count reasoning tokens, but not input tokens or rejected speculative draft tokens.
- Utilization and contract rates change the answer linearly. Half as much productive time means half as many tokens; a 30% lower rental price buys about 43% more tokens. A $1M negotiated quote may differ materially from list prices.
- This is rental cost, not GPU ownership. Standard host resources and electricity are within the instance charge; engineering, orchestration, extra storage, egress, taxes, validation and filtering are excluded. Apply a separate budget reserve for those costs.
- No fixed completion date. Spending $1M on one $7/hour GPU would take about 16 years. In practice use many independent replicas to spend the budget faster, subject to available capacity and similar efficiency.
- Compare the same job. The archived table’s API column follows the input/output ratio at the top, while these hypothetical GPU assumptions use an output-heavy workload, not the measured 8:1 benchmark workload. Set API input/output ratio appropriately; real cached-input and batch discounts may further change the comparison. Equal generated-token counts do not imply equal useful math solutions.
Bottom line: Qwen 27B has the clearest self-hosting upside in these scenarios. For the enormous models, and especially DeepSeek’s inexpensive off-peak API, renting GPUs is not automatically cheaper. These estimates support a benchmark plan—not a $1M purchase decision.