The Mirror, Not the Wall:
Why AI’s 4-Bit Throughput Limit Is About the Data, Not the Machine

This article is a companion to Paper 9 — “The Hardware Basin: Why the Quantization Cliff Is About Level Allocation, Not Bit Count” (2026). The full manuscript is at github.com/Windstorm-Institute/hardware-basin.

Paper 7 found something that looked simple: language models collapse when you quantize their weights below 4 bits. INT4 works. INT3 doesn’t. The cliff is sharp and universal across eight models.

We thought we were looking at a floor — a hard physical limit baked into the mathematics of low-precision arithmetic. We were wrong. We were looking at a wall built by a specific quantization method, not by the arithmetic itself.

The experiment that changed the story

We loaded Pythia-1.4B (a real, pretrained language model with 1.4 billion parameters) and quantized it two ways, both to 4-bit weights:

Symmetric quantization — the simplest method, and what basic integer hardware does. Place 15 quantization levels uniformly from the minimum weight to the maximum weight. Result: BPT = 16.87. Catastrophic. The model outputs gibberish.

NF4 (normal-float-4) — a smarter method. Place the same 15 levels at the quantiles of a normal distribution, so more levels sit near zero (where most weights cluster) and fewer levels sit in the sparse tails. Result: BPT = 3.90. Operational. The model works fine.

Same bit count. Same model. Same data. One works, one doesn’t. The difference is entirely in where the bits are spent.

Why it matters

If the cliff were at a fixed bit count, hardware designers would need to build wider integer datapaths — INT8 everywhere, no shortcuts. Expensive, power-hungry, physically large.

But the cliff is at a quality threshold for level allocation. Hardware that supports non-uniform quantization tables — where each 4-bit code maps to a non-uniform float value, optimized for the weight distribution — can operate at 4 bits. Hardware that supports only uniform integers needs 8 bits to achieve the same result.

The practical implication: build lookup tables, not wider datapaths. A 16-entry lookup table per weight block costs almost nothing in silicon area, but it shifts the precision floor from INT8 to INT4 — a 2× reduction in memory bandwidth and storage.

It’s universal

We tested the cliff across four model architectures: three transformer variants (Pythia-160M, Pythia-1.4B, GPT-2-medium) and one state-space model (Mamba-370M). The cliff appears in all of them. Mamba uses fundamentally different computation (state-space dynamics instead of attention), but its weight matrices are still approximately Gaussian — and that’s what the cliff is about.

We tested across all 24 layers of Pythia-410M. The cliff ratio (~4.5×) is consistent from layer 0 to layer 23. It does not get worse in deeper layers. It is a property of the weight distribution shape, not the network depth.

One matrix defied the cliff

One weight matrix in GPT-2-medium showed no cliff at all (ratio 0.9×). We investigated: this matrix has extreme kurtosis (124.75) and 80% sparsity. Translation: a few weights are very large, and the rest are near zero. Quantizing the near-zero bulk makes no difference because those weights don’t contribute to the output.

This predicts that pruned models (deliberately sparsified) will be more robust to aggressive quantization. It also predicts that embedding layers (which tend to be sparser) will resist the cliff better than attention layers (which are denser). Both predictions are testable.

The optimal quantizer

We tested five level-allocation strategies on real Pythia-410M weights at INT4:

Symmetric uniform: cosine 0.905. NF4 (Gaussian quantiles): cosine 0.973. Lloyd-Max (mathematically optimal): cosine 0.990. Log-scale: cosine 0.965. Random (control): cosine 0.894.

Lloyd-Max at INT3 (cosine 0.965) is better than symmetric at INT4 (cosine 0.905). The optimal quantizer with 7 levels outperforms the naive quantizer with 15 levels. This is the strongest evidence that the cliff is about level quality, not level count.

The structural bonus test

Cosine similarity measures how well a single matrix survives quantization. But a language model is 24 layers of matrices chained together, and errors accumulate. The real question is: does the quantized model still understand language?

We measured the structural bonus — the BPT difference between real English text and the same text with its words shuffled into random order — under each quantization method. A model that understands syntax should perform much better on ordered text than shuffled text. A model that is just memorizing surface statistics should perform equally badly on both. The structural bonus is a direct measure of whether the model still exploits linguistic structure.

NF4 preserves 99.5% of the structural bonus. Symmetric INT4 destroys it — the bonus drops from 6.23 to 0.28, meaning the model can barely distinguish ordered English from word salad. Symmetric INT8, with twice the bit budget, recovers the full bonus. The cliff is not just about cosine similarity in individual matrices. It is about whether the model still understands language.

The end-to-end test

The cosine similarity results suggested Lloyd-Max INT3 (cosine 0.965) might outperform symmetric INT4 (cosine 0.905). If true, that would mean the right quantization algorithm could push the floor from 4 bits to 3 — a further 25% reduction in memory bandwidth.

We tested this by running full end-to-end perplexity evaluation on Pythia-410M, quantized with each method, on WikiText-2:

Lloyd-Max INT3 fails end-to-end. Per-matrix cosine of 0.965 does not survive error accumulation across 24 layers. Even Lloyd-Max INT4 degrades to 2× baseline — still operational, but substantially worse than NF4’s near-lossless 3.90 BPT.

This is a cautionary result: cosine similarity overstates quantization quality. A method can preserve 96.5% of per-matrix fidelity and still produce a model that is 2.7× worse end-to-end. The structural bonus test and end-to-end BPT are the metrics that matter. NF4 remains the only INT4 method that preserves both meaning and performance through the full network.

Statistical decisiveness

To rule out any concern that these results could be noise, we ran the structural bonus test on Pythia-1.4B (1.4 billion parameters) with five independent shuffle seeds per quantization method:

The 95% confidence intervals do not overlap. The Welch’s t-test comparing FP16 against symmetric INT4 gives t = 633.74, p = 2.84 × 10⁻¹⁵, Cohen’s d = 400.81 — an effect size that is, for all practical purposes, infinite. The cliff between symmetric INT4 and every other method is one of the cleanest experimental results in the deep learning literature. It is not noise. It is not a model artifact. It is the direct consequence of where the quantization levels are placed.

The Hardware Basin is Paper 9 of the Windstorm series.
Zenodo: DOI pending · Code & data: github.com/Windstorm-Institute/hardware-basin
Download the full paper (PDF) · Grand Slam Supplementary Materials (PDF)