In 1948, a Bell Labs engineer named Claude Shannon published a paper that quietly created the modern world. He asked a question nobody had asked precisely before: how much information is in a message? Not "what does it mean" or "is it important" — how MUCH is there, in numbers, the way you'd measure a length in meters or a weight in kilograms. His answer is the reason your phone, your WiFi, and every file on your computer work the way they do.

This is the hub article for a five-part cluster on information and bits. The other four go deeper into each piece: Shannon entropy explained works through the famous H = −Σ p log₂ p formula with coins on the table; how compression works shows how ZIP files and MP3s squeeze data toward Shannon's limit; error-correction codes explained covers how CDs, QR codes, and space probes survive noise; and why noise limits communication explains the hard speed limit on every channel, from telegraph wires to 5G.

The wrong question and the right one

Suppose I send you a message. How much information did you receive?

Your first instinct is probably to think about meaning. A love letter feels like it carries more than a grocery list. But meaning is a swamp. It depends on who reads the message, what they already know, what they care about. There's no ruler for it.

Shannon's move was to throw meaning out entirely. He wrote, right in the paper, that the semantic aspects of communication are "irrelevant to the engineering problem." What he measured instead was this: how much uncertainty does the message remove?

Before the message arrives, you don't know what it will say — that's the whole point of sending it. There is a set of things it could say, each with some probability. When it arrives, that uncertainty collapses. The amount of information in the message is the amount of uncertainty that just disappeared.

A message that tells you something you already knew for certain carries zero information. "The sun rose this morning" — zero. You'd have bet your house on it. A message that settles a genuine 50/50 question — "the coin came up heads" — carries exactly one unit. And that unit has a name.

A bit is one yes/no question's worth of surprise

The bit is the natural unit of information: the amount you learn from the answer to one fair yes/no question. Fair meaning that before the answer, you'd have given even odds either way.

One coin flip: one bit. Two coin flips: two bits. And here's where it gets useful — suppose I'm thinking of one of four equally likely things: north, south, east, west. How much information do you get when I tell you the answer?

Count the yes/no questions you'd need. "Is it north or south?" (one bit). "Okay, of those two, is it north?" (a second bit). Two questions pin it down, so the answer carries two bits. Eight equally likely options need three questions: three bits. Sixteen options, four bits.

You can see the pattern: for N equally likely possibilities, the information in learning the answer is log₂ N — the number of times you can halve the possibilities until only one remains. That's all the logarithm is doing here. It's not fancy math; it's counting halvings.

This is also why twenty well-chosen yes/no questions can identify roughly a million things (2²⁰ is about a million) — the entire trick behind the game Twenty Questions.

Surprise is the currency

Now the crucial refinement. What if the possibilities aren't equally likely?

Suppose you live in the Atacama Desert and I text you each morning: "no rain today." That message is almost certain in advance, so it carries almost no information. But one morning the text says "rain." That message is a shock — and it carries a lot of information, precisely because it was improbable.

Shannon quantified this: an event with probability p carries log₂(1/p) bits of surprise. The rarer the event, the bigger the number. A 50% event carries 1 bit. A 1-in-a-million event carries about 20 bits. A 100% certain event carries log₂(1) = 0 bits — no surprise, no information.

Average that surprise over all the messages a source can produce, weighted by how often each occurs, and you get the source's entropy: the average information per message. That average is the star of the show, and Shannon entropy explained walks through it with actual coins.

Notice what this framework does NOT care about: whether the message is true, useful, beautiful, or in English. A page of random letters has more Shannon information than a page of Shakespeare, because Shakespeare is partly predictable — English has patterns, and patterns mean you could have guessed some of it. That sounds like a defect until you realize it's the whole point. If parts of a message are predictable, you don't need to transmit them in full. That single observation is the foundation of every compression algorithm ever written.

Why measuring information changed everything

Once information is a number, you can prove things about it. Shannon proved two theorems in that 1948 paper, and between them they define the boundaries of all digital communication.

The source coding theorem: a source with entropy H bits per symbol cannot be losslessly compressed below H bits per symbol on average — but you can get as close to H as you like. Entropy is the incompressible core of a message. ZIP, PNG, and FLAC are all engineering assaults on that limit.

The noisy channel coding theorem: every noisy channel has a capacity C, measured in bits per second. Below that rate, you can communicate with as few errors as you like — not by shouting louder, but by encoding cleverly, adding structured redundancy so the receiver can detect and repair the damage noise causes. Above C, reliable communication is mathematically impossible. Not hard. Impossible. The formula for C on a real channel, and what it means for your WiFi, is the subject of why noise limits communication.

The astonishing part of the second theorem is that noise doesn't make communication unreliable — it just makes it slower. That was not obvious in 1948. Engineers assumed that a noisy line meant garbled messages, full stop. Shannon showed the garbling can be engineered away completely, at a known, calculable cost in speed.

Information is physical

One more idea worth carrying with you: information isn't an abstraction floating above the world. It always lives in some physical arrangement — ink on paper, charge in a transistor, the spin of an electron, a pulse of light in a fiber. Erasing a bit has a minimum energy cost (Landauer's principle, roughly 3 × 10⁻²¹ joules per bit at room temperature — tiny, but not zero). Black hole physicists argue about where information goes when matter falls in. The connection between Shannon's entropy and the entropy of steam engines is real mathematics, not a pun — though the two ideas are aimed at different questions, as the entropy article in this cluster spells out.

The way a single 1948 insight quietly sits underneath phones, files, and space probes is exactly the kind of thing that rewards a few minutes of curiosity — it's the sort of everyday-technology story that apps like NerdSip serve up in five-minute portions, and once you've seen it, you can't look at a loading bar the same way.

The takeaway

Information is removed uncertainty, measured in bits, where one bit is one fair yes/no question's worth of surprise. Improbable messages carry more information than expected ones; perfectly predictable messages carry none. From this one definition, Shannon derived hard limits — how far data can be compressed, how fast a noisy channel can carry it — that no cleverness can beat, and that all of our technology has spent 75+ years learning to approach. The four supporting articles in this cluster each take one of those threads and pull.