The four-bit basin is a property of the data, not the model

The plain-language companion to "The Throughput Basin Origin" (Zenodo, 2026), published — as always — alongside our own attempt to tear it apart. Full works below.

For six papers running, we'd been circling something strange. Every AI language model we measured — big, small, this lab's, that lab's, this year's, last year's — settled into the same narrow band of about four bits of real information per chunk of text. Not three. Not seven. Four, give or take. We named it the throughput basin, and like anyone who finds a suspiciously flat spot in a landscape that ought to be jagged, we couldn't stop asking: why is it there?

For a long time the honest answer was "we haven't the foggiest." We had guesses, and the spookiest one kept us up at night: maybe four bits was a hidden law of thinking machines — a ceiling baked into their very architecture, the way a top speed is baked into an engine. If that were true, it'd be one of the deepest facts anyone had ever found about AI. Paper 7 is the experiment that finally let us check. And it turned out we'd been fooled by our own ruler.

The Trouble With Weighing Buckets

Here's the catch hidden in a number like "four bits per chunk." It's secretly measuring two things at once, and you can't pry them apart by squinting harder. It measures the machine — how much it can carry. And it measures the text — how much is actually in there to carry.

If the only thing you ever weigh is buckets of water, you'll eventually declare that buckets weigh about eight pounds. Perfectly true. Also completely misleading — because the bucket was never the point. The water was. For six papers, we'd been weighing buckets of water and writing papers about the weight of buckets.

So We Changed the Water

We built a text from scratch — not scraped from the web, not filtered, but engineered — with exactly double the information density of ordinary English. Twice as much stuff crammed into every chunk. Then we fed it to the very same AI that had always, dependably, flattened out at four bits on normal text. Nothing about the machine changed. Only its diet.

It didn't stop at four. It climbed — sailed right past the basin where every previous model had plateaued, and tracked its richer diet almost perfectly. And here's the clincher: when we did it again with a machine fourteen times bigger, it landed on the exact same richer number. Making the machine bigger didn't crack the ceiling, because there was no ceiling in the machine. The earlier models hadn't been hitting a wall. They'd been hitting a surface — the surface of the language we'd been feeding them.

Ordinary English, it turns out, is gloriously repetitive. Most of what comes next in a sentence is already half-promised by what came before. That redundancy has a number, and the number is about four bits, and that's the number we kept dutifully measuring, model after model, because that's the number that was in the room. The AI was a mirror. We'd spent six papers measuring the wall behind it.

Why This Flips Everything Sideways

It'd be easy to shrug at this. So the limit's in the data — so what, the data is what we've got. But that shrug is exactly wrong, and it's wrong in a way that's worth real money.

If you believe the limit lives in the machine, you spend your years — and your billions — building bigger machines, fancier architectures, exotic new designs, drilling furiously through a wall that isn't holding anything up. But if you believe the limit lives in the data, the whole problem swivels ninety degrees. Now the interesting question isn't "how do we build a smarter model?" It's "what would it take to put richer stuff in front of the one we already have?" Those are wildly different quests, funding wildly different labs. Paper 7 doesn't crown a winner — it just points out that the field's been quietly mixing up two questions that were never the same question.

The Part Where We Rat on Ourselves

This is where a normal article would slap a bow on it and link the PDF. We're going to do something else first, because it is quite literally the institute's entire reason for existing.

Before we published a word of this, we sicced an in-house review on our own experiments — a ruthless internal skeptic whose only job was to comb the data and catch every place we'd quietly oversold. It found eight problems. Four were serious enough to say, flatly: your headline is not yet earned by these experiments. So we published that review right next to the paper, warts and all — not six months later in some rival's rebuttal, but in the same breath as the claim itself.

Then we went and fixed them. One turned out to be a data leak. One was a bookkeeping error in how we fed text to the model. And one taught us something genuinely important that we'll come back to in a second. All four are now resolved, and the headline — the basin is driven by the data, not the machine — came out the other side standing, and stronger for the beating. We tell you this because at the Windstorm Institute, the attempt to kill an idea is supposed to arrive holding hands with the idea. That's not a footnote. That's the whole job.

The Ruler Was Lying, Too

Here's the lesson buried in that self-review, and it's arguably bigger than the main result. At one heart-stopping moment, our biggest model came back reading four bits on the rich text instead of the eight we expected, and for twelve hours we thought we'd demolished our own thesis. The automated report cheerfully announced: "THESIS FALSIFIED."

It hadn't been. The number was an illusion created by how we were chopping the text into chunks. Chop it into bigger chunks and the "bits per chunk" number mechanically shrinks, even though the machine is extracting exactly the same total information. Measure it properly — bits per letter, which doesn't care how you chop — and the illusion vanished: eight bits, at both sizes, identical. The uncomfortable moral: "bits per chunk," the yardstick the entire field leans on (us included, for six papers), is partly a measure of the packaging, not the contents. We're saying that out loud precisely because it comes back to bite our own earlier work, not just other people's.

The Mirror Is Smooth

A fair skeptic could still ask: sure, but maybe something weird lurks right at four bits — some secret attractor that only shows up in that exact zone. So we built texts at five, six, and seven bits and checked. Every one tracked its diet cleanly, no kinks, no plateaus, no hidden pull toward four. The mirror is smooth all the way through the zone where the basin lives. If there were a wall in the machine, these would've smacked into it. They didn't.

And the richness of the food isn't the whole story either — structure matters too. Feed the model text with grammar-like nesting and it compresses some of that away, landing between the flat-data ceiling and the natural-language floor. So the real rule is elegantly simple: what an AI carries per step equals how much is in the text, minus however much of it the model can cleverly predict away. Natural language sits at four bits not because of some silicon law, but because after you subtract everything a good reader can anticipate, four bits of genuine surprise is about what's left.

The Mirror

There's a moment in almost every long hunt when the thing you've been studying turns out to be the thing you've been studying with. You spend years measuring something, and slowly realize you've been measuring your own ruler.

That's what Paper 7 was for us. The four-bit basin was real, reproducible, robust across every model and scale we threw at it — and it earned every ounce of attention we gave it. But it was never a property of the machines. It was a property of the words those machines were swimming in, reflected back so faithfully that for years it looked like a law of nature. It was a mirror. We mistook it for a wall.

The good news about mirrors is that once you know they're mirrors, you can step around them. The road to more capable AI doesn't run through some new architecture or exotic new physics — it runs through richer food: vision, sound, the texture of the physical world. Because language sits at four bits for a reason we've now chased all the way to its source — that's about all the biological creatures who invented it could ever pour in. To go further, you don't need a better mirror. You need something new to reflect. That "something new" is exactly where Papers 8 and 9 go next: into the eye, the ear, and the silicon chip itself.

The Throughput Basin Origin is Paper 7 of the Windstorm series.
Zenodo (concept DOI, always-latest): 10.5281/zenodo.19498582 · Current version v1.6 (April 2026): 10.5281/zenodo.19498582 · Code & data: github.com/Windstorm-Institute/throughput-basin-origin
Download the full paper (PDF) · Grand Slam Supplementary Materials (PDF)