One Division Eliminated Half the Model Candidates
Measured DGX Spark memory bandwidth at 231GB/s. Divide by active parameters and you get the decode ceiling; every dense model was eliminated by arithmetic alone
The DGX Spark is the box I bought to run local LLMs on. Having it search for something better than the model currently loaded only made the candidate list longer, and alongside it the complaint piled up that the performance was poor for what the hardware cost. So before picking any more models, I decided to measure why it was slow.
What I set out to do
From the day I came back I kept having it search for “is there a better model than the Gemma 4 26B-A4B we’re using now.” The candidate list grew. 27B, 33B, 35B, 122B. By name alone they all looked good.
Disappointment piled up in the meantime. On the night of Aug 18 I just threw it out there.
For how absurdly expensive this hardware is, the performance it shows running an LLM is really bad.
Claude Code’s answer wasn’t consolation. It was numbers. It showed by calculation that the complaint was justified.
How I asked
I told it to measure why it was slow. The memory bandwidth measurement came first. 231GB/s. That’s 85% of the 273GB/s spec. The GB10 is Blackwell compute units bolted to laptop-class memory. The number I saw when buying was 1 PFLOP; the number that binds when using it was 231GB/s.
LLM decode reads every active parameter from memory once per generated token. So the ceiling comes out of a single division.
decode ceiling (t/s) = 231GB/s ÷ (active parameter count × bytes per parameter)Where it broke
The problem was that I was only doing this calculation now. Back when I loaded a 31B dense model I hit a wall around 7t/s. At the time I assumed it was a config problem and moved on.
Load a dense 27B in BF16 and you read 54GB per token. The ceiling is 4.2t/s. No setting gets above that. The Gemma 4 26B-A4B we use now: the A4B in the name means 4B active. It reads only 8GB per token, so the ceiling is 29t/s. Same 26B class, seven times the ceiling.
This one calculation cut the candidate list in half. The rule is simple. If the name has no active-parameter marker like A4B or A3B, drop it. Dense models like Qwen3.8-27B, EXAONE-4.5-33B, and Muse-Glimmer-30B were out before I even looked at benchmark scores. Things like Kimi-K3 (2.78T) and GLM-5.2 (753B) exceed 90GB even squeezed to 4-bit and don’t fit in memory. The only survivors are MoE models with 10B or fewer active parameters.
How I fixed it
The same division also showed this hardware’s strength. Prefill is compute-bound, not bandwidth-bound, so the Blackwell numbers apply. Measured at about 1,900tok/s. It swallowed a 116,489-token document in 64.9 seconds. Hardware that reads 110k tokens in a minute is not common at this price.
So the shape of what this machine does well was settled. Long input, short output. Document work like disclosure summaries with a 15:1 input-to-output ratio is the strength zone; chat that produces long free-form prose is the weakness zone.
There is only one way past the decode ceiling. Speculative decoding reads memory once and verifies several tokens, which effectively increases the numerator of the division. This calculation explained why turning on MTP the day before had made the real workload more than twice as fast.
Lessons
- Do the division before picking a model. Bandwidth ÷ (active parameters × bytes). A model that can’t clear this ceiling doesn’t need its benchmarks read.
- Look at active parameter count, not parameter count. Only MoE models with A4B, A3B in the name get meaningful speed on this hardware.
- Spec-sheet numbers and the numbers that bind in use are different. Bandwidth measured at 85%, and compute performance is barely used in decode.
- Fit the work to the hardware’s strength zone. Prefill 1,900tok/s, decode around 30t/s. Long input, short output is the fit.
- Measure “why is it slow” and the complaint turns into arithmetic. The disappointment stays, but what to do next becomes clear.