Pick a model, see which desk-side machines can actually hold it and how fast they decode. Throughput is derived from memory bandwidth and compute, not typed in by hand — the formula is at the bottom and is calibrated against published measurements. Click headers to sort.
Decode is memory-bandwidth-bound: tok/s ≈ eff × bandwidth ÷ active-weight-bytes.
eff is 0.65 for dense models and 0.42 for MoE (gather overhead and poor locality).
Calibrated against two published measurements: M3 Ultra 819 GB/s on DeepSeek-R1 671B-A37B Q4
lands at ~19 tok/s, and DGX Spark 273 GB/s on gpt-oss-120B MXFP4 lands at ~39 tok/s —
both within a few percent of reported figures.
Prefill is compute-bound: tok/s ≈ eff × FLOPS ÷ (2 × active params), with a much
lower software-maturity efficiency on non-CUDA stacks (0.35 CUDA, 0.15 Metal, 0.10 ROCm/oneAPI/Tenstorrent).
Prefill is the softest number here — it moves 2–3× with the runtime, quantization and kernel.
Fit assumes weights × 1.15 for KV cache and activations at moderate context, against 92% of VRAM
on discrete cards or 75% of unified memory on Macs (macOS won't hand the GPU all of it by default).
Batch 8 is not 8× decode: batching amortizes weight reads, so it scales well on high-FLOPS cards
and poorly on bandwidth-rich/compute-poor ones. Modelled as decode × min(8, 1 + FLOPS headroom).
Prices are typical US street or config prices as of late Aug 2026, cards only where noted —
a 5090 or RTX PRO 6000 still needs a CPU, board, PSU and case on top. The 2026 GDDR/DRAM shortage has
moved several of these well above launch MSRP, and they keep moving.
Estimates, not lab certificates. Every number above is a model output, so treat rankings as sound and absolute values as ±30%. Real throughput depends on runtime (llama.cpp / MLX / vLLM / TensorRT-LLM), quantization, context length and thermal headroom.