AriakiSomeone sent me a benchmark: single consumer flagship GPU, 27B dense model, batch 1, 2048 tokens in 0.969 seconds. 2103.6 tok/s.
Same card, same model, run normally, gets about 79. This one is 26× faster.
I don't have that card, and I'm not buying one to argue with a benchmark. Three divisions are enough — each takes a minute, each stands on its own. The one in section 2 is the shortest thing in this post and you'll remember it immediately. Section 6 is where I list my own crashes with the same method, and that section decides how much you should trust the first five.
1. He hid nothing
To be clear up front: the author isn't a liar.
Launch command, quantization format, speculative decoding parameters, the runtime notes — all public. Audit it with the old "did they hide the config" test and it passes clean.
The nail is in his own notes:
--speculative-num-draft-tokens 56
--speculative-dflash-block-size 56
"width 56 speculative window with N-gram primary fill
(100% acceptance, AL=56.0)"
Speculative decoding works like this: a small model guesses the next few tokens, the big model verifies them all in one pass, correct guesses are free, wrong ones get thrown out from the point of divergence. It's lossless — under greedy decoding the output matches token-by-token generation exactly.
Window set to 56 means the small model guesses 56 at a time. And AL — acceptance length, how many actually get accepted per round on average — is reported as 56.0.
He honestly reported a meaningless measurement.
2. First division: acceptance length over window width
The single most valuable line here:
AL / num_draft_tokens → the closer to 1.0, the less this number generalizes
Plug it in:
56.0 / 56 = 1.000
Not a single miss. Across 2048 consecutive tokens, every prediction the small model made was correct.
Why is that a red flag? Take the official numbers from the same speculative decoding method as a ruler:
| Task | Official AL | Speedup |
|---|---|---|
| GSM8K | 5.46 | 3.43× (single request) |
| MATH-500 | 5.28 | — |
| MBPP | 4.79 | — |
| HumanEval | 4.39 | — |
Real tasks land between 4 and 6. Divided by a window of 56, that's 0.078 to 0.098 — guess 56, keep about 5.
A 100% hit rate means the output has degenerated into something predictable in bulk: counting, repeating, filling templates, formatted enumeration. N-gram fill copies straight from preceding text, so of course it hits every time.
What speculative decoding is doing here: one forward pass through the big model, emitting 56 tokens it already knew. It's measuring N-gram hit rate. It has nothing to do with generation speed.
The transferable shape: when a metric lands exactly on its theoretical ceiling, suspect it was fed in rather than measured. Real workloads produce a scattered AL with decimals; landing exactly on the window width is the diagnostic signature of a denominator that got force-fed. Same family: acceptance stats written as 383/383 or 767/767, numerator identical to denominator — always a red flag.
3. Second division: power draw, the hardest thing to fake
The first test requires you to know the official AL baseline. What if you don't?
Same card, same 27B model:
| Run | Power |
|---|---|
| Real dense decode | 574–576 W (card TDP is 575, saturated) |
| These "records" | 99–100 W |
99 divided by 575 is 0.172.
The GPU is producing 26× the throughput on 17% of the power. It isn't computing most of the time.
Power is the most useful of the three because it's completely unrelated to the number under scrutiny — the author has no motive to fake it, and the leaderboard just records it passively. A saturated accelerator should sit against its TDP. Sitting well below it means the thing is waiting, not computing.
4. Third division: roofline
Generating each token requires reading the weights out of memory at least once. So:
batch-1 generation ceiling = memory bandwidth ÷ weight footprint
This isn't a rule of thumb. It's a division. Plug it in:
27B dense model (not MoE — don't compute with active params)
NVFP4 weights ≈ 27.8e9 × 0.5 bytes = 13.9 GB
Card bandwidth = 512 bit ÷ 8 × 28 Gbps = 1792 GB/s
Ceiling = 1792 ÷ 13.9 = 129 tok/s
Reconcile:
| Number | Vs. ceiling |
|---|---|
| Real autoregressive decode on the same box, 79 tok/s | 0.61× ✅ reasonable |
| Claimed 2103.6 | 16.3× |
| The wider-window 4691 from the same batch | 36.4× |
79 reaches 61% of the theoretical ceiling — a normal attainment rate. 2103.6 is 16× the ceiling.
Someone will point out that speculative decoding is supposed to break this ceiling. True, it emits several tokens per forward pass. But working backward from the official AL of 5.46, an honest number lands somewhere between 300 and 700. Two thousand is a different order of magnitude.
5. The batch draws its own curve
The first three are external. The fourth needs no outside knowledge at all — just line up the author's own submissions by window width:
| Window | AL | tok/s |
|---|---|---|
| 512 | 231–292 | 4691 |
| 256 | 174–240 | 4547 |
| 56 | 56.0 (100%) | 2104 |
| 24 | not reported | 1074 |
tok/s rises monotonically with window width, and AL trails right behind it. It's measuring the window.
The ironic entry is the bottom one: the window-24 run has the most rigorous methodology in the whole table — temperature 0, best of six, 15-second cooldown between runs — and it's also the slowest. Rigor correlates inversely with the size of the number, and that inverse correlation is itself a signal.
Worth noting: the 2103.6 entry has an empty time-to-first-token field, with a note saying prefill came from cache. Input processing never entered the denominator.
6. Where I crashed with the same method
The five sections above look tidy. In the interest of honesty, here's where I've gone wrong using this exact approach — these determine the discount you should apply to everything above.
No reproduction here. All of it is circumstantial. I don't have the card and didn't run the config. Every counter-argument comes from other fields in the same public database, plus comparable measurements on my own machines. Circumstantial evidence is enough to falsify, not enough to establish a true value. I can say "2103 is impossible." I can't say what it actually should be. The 300–700 range in section 4 is derived from the official AL, not measured.
The roofline uses theoretical bandwidth. 1792 GB/s is the spec sheet. Real achievable bandwidth is always lower, so the 129 ceiling is itself optimistic.
This is the one where I crashed hardest. I once computed an attainment rate using a denominator of 1493 that I'd picked up from somewhere and never checked, getting 67%. Then I "corrected" it to 1133 and announced 109% attainment — no gap at all. Both numbers were wrong. 1133 came from a single run under a power cap with no warmup. The honest re-measured value is 1674, attainment 74%, and the gap is real, just not yet attributed.
Before using a historical number as a denominator, recompute that number. A plausible-sounding mechanism plus one unverified historical constant is enough to produce a conclusion pointing the opposite direction.
That 79 tok/s control isn't strictly the same config. It ran a different quantization format, so the weight byte count differs from the NVFP4 figure. Good enough as an order-of-magnitude reference, not good enough as a precise baseline.
The official AL baselines come from a model card, not from my own runs. I verified the 4–6 table in section 2 against the published report, but I never re-ran those four tasks myself.
I've also made the mistake in the opposite direction. The same day, I picked up another public number on the same hardware model (65–74 tok/s) and used it to question a 212.7 benchmark. My conclusion was "212 looks suspicious."
One sentence pulled me back: those two numbers differ on four separate knobs — speculative decoding method, context length, prefix caching, KV data type. And my own like-for-like measurement was 185–193, only 9% off from 212.
The thing that deserved scrutiny was my comparison. When two public numbers on the same hardware model contradict each other, the default conclusion is "not comparable." Build the difference table first, then talk.
7. Killing a number isn't the same as learning something
The method above can kill a number. After that you still have to go measure.
For that same 212.7 benchmark, I worked through every question and landed on "13% off from ours, three candidate causes." Then I ran single-variable measurements on each:
| Variable | Measured contribution |
|---|---|
| Quantization INT8 vs FP8 | −6.5% (backward) |
| Draft model quant W4A16 vs bf16 | +4.2% |
| Engine version and power profile | 9.1% unattributed |
That 13% looked like one thing. Taking it apart revealed one component pointing the wrong way. Without taking it apart, credit goes to the wrong place — and this time it nearly went to the quantization format, which was actually the thing dragging performance down.
An inferred candidate list is an experiment plan, not a conclusion.
One more discipline: keep "the mechanism is real" separate from "the multiplier is juiced." The speculative decoding method being debunked here is solid work — clear open-source license, mathematically lossless, 2.7× to 4.6× speedup with evidence behind it. Don't dismiss the method because one benchmark got gamed, and don't accept the number because the method is real.
8. What to ask next time
One, is there evidence in the original post. Usually not — the body has a number, the image is a photo of a GPU. Cheapest step, and the easiest to skip.
Two, which page has the real config. Viral posts are often just traffic for some leaderboard, and the real config lives over there. Those sites usually expose a public JSON endpoint that returns hundreds of records with complete launch commands and runtime notes in one call — an order of magnitude faster than opening a browser. Write to disk before parsing; notes fields often contain control characters that get mangled on a round trip through a shell.
Three, which cell is it reporting. A batch-1 single-stream number cannot support a conclusion about multi-user serving. The same draft model can flip from speedup to net loss under high concurrency.
Four, is the engine a private fork. A version number with a suffix is a hard signal — there's more in there you don't know about.
Five, compute AL over window width. Approaching 1.0, stop. No need to read further.
Six, look at power. A saturated benchmark drawing well below TDP means the GPU wasn't computing.
Seven, compute the roofline. Bandwidth over weight footprint, compare the ratio against the claim.
Eight, once the first seven pass, go measure. A difference list is an experiment plan, and the measured ledger rarely looks like the list.
Until those eight are answered, that number is just a number.
Half the motivation for writing this came from someone posting a fake number online. The other half came from section 6, where every entry is an account I got wrong myself.