A child thinks of a whole number between 1 and a million and dares you to find it. You may ask twenty yes-or-no questions. She answers honestly every time. Can you corner the number?
You can, and most readers already feel how: halve the space each time. "Is it bigger than five hundred thousand?" — one answer, and a million possibilities become five hundred thousand. Nineteen more halvings and exactly one number remains, because Chapter 1's ladder says twenty doublings make a million: \(2^{20} = 1{,}048{,}576\). Twenty yes-or-no answers — twenty bits — locate one choice out of a million.
That is the deepest thing a bit ever gets: a bit is an answer that cuts the possibilities in half. This chapter turns that game into a measure — a way to say, in bits, how much surprise a choice contains — and the measure turns out to be the one Bitcoin's security is written in. The word for it is entropy.
The twenty-questions trick works because each answer splits the remaining space in two. One answer: 2 possibilities. Two answers: 4. Twenty answers: over a million. The ladder from §1.3 runs the whole game — \(n\) answers distinguish \(2^n\) things, no more, no less.
Flip the game around and it becomes a measure. If a choice can be cornered by twenty yes-or-no questions and no fewer, the choice carries twenty bits of picking-out. A coin flip's outcome: 1 question, 1 bit. A die's outcome: between 2 and 3 questions' worth — the ladder says \(2^2 = 4\) is too few faces and \(2^3 = 8\) is more than enough, so a die carries more than 2 and less than 3 bits. The exact figure needs one refinement, arriving in §4.3.
Notice what the measure is made of: the number of halvings the choice is worth. A choice from a bigger fair space is more bits of surprise, by exactly the ladder's count.
Make the measure official for the clean case — every outcome equally likely:
A choice made uniformly from \(2^n\) possibilities carries exactly \(n\) bits of entropy. It is the number of fair coin flips the choice is worth — the number of halvings needed to corner it.
The card on the table is now measurable with zero new machinery. Twelve words from 2,048, uniformly chosen: Chapter 1 counted \(2{,}048^{12} = 2^{132}\) phrases, so the phrase carries 132 bits of entropy. Each word is 11 bits — eleven halvings — and twelve of them multiply the space, so the bits add: \(12 \times 11 = 132\). Entropy in bits is just the doubling ladder read as a measure of choice.
One word of vocabulary, for the record. The number of doublings needed to reach a number \(x\) — what power of 2 it is — has a name: the base-2 logarithm of \(x\). This book almost never needs the name, because the ladder picture ("how many halvings?") always suffices. Where a machine must compute it, Python's math.log2 does the work: math.log2(2048**12) prints 132.0.
The fair-space definition covers coins and fair dice. Real choice processes are usually unfair — and unfairness changes the surprise. Consider a loaded four-sided die rigged to show its faces with these chances:
1: \(\tfrac{1}{2}\) 2: \(\tfrac{1}{4}\) 3: \(\tfrac{1}{8}\) 4: \(\tfrac{1}{8}\)
A roll of 1 carries little surprise — it happens half the time; a 50–50 event is one flip's worth, 1 bit. A roll of 3 is an 8-to-1 long shot: three halvings, 3 bits. The rule: an outcome with chance \(p\) carries the flip-count of \(1/p\) bits of surprise — halving the chance adds exactly one bit of surprise. (For chances that are exact powers of 1/2, like these, the counts are exact: \(1/2 \to 1\) bit, \(1/4 \to 2\) bits, \(1/8 \to 3\) bits.)
So the faces of the loaded die deliver different surprises per roll. What is the surprise of the die as a process? Average it — Chapter 3's expected value, each face's surprise times its chance:
\[ \tfrac{1}{2} \times 1 \;+\; \tfrac{1}{4} \times 2 \;+\; \tfrac{1}{8} \times 3 \;+\; \tfrac{1}{8} \times 3 \;=\; 1.75 \text{ bits}. \]A fair four-sided die, every face 2 bits of surprise, averages exactly 2 bits. The loaded die averages less. Unfairness leaks surprise: the rigged die is more predictable, and the entropy says by how much — a quarter of a bit per roll.
And the fair six-sided die from §4.1 gets its exact figure at last: each face carries the flip-count of 6 — math.log2(6) — about \(2.585\) bits per roll, between 2 and 3 as promised.
The entropy of a choice process is its expected surprise: for each outcome, multiply its chance by the flip-count of \(1\) over that chance, and sum over all outcomes. For a fair space of \(2^n\) outcomes the result is exactly \(n\) bits, matching Definition 4.1. Among all processes with the same number of outcomes, the fair one has the highest entropy; every bias lowers it.
Claude Shannon built this measure in 1948 at Bell Labs, in a paper called A Mathematical Theory of Communication — the founding document of information theory, and the paper that put the word bit (coined a year earlier by his colleague John Tukey, squeezing "binary digit") to work. Shannon was measuring how much a message says; the same number measures how much a choice surprises, because a message and a choice are the same kind of object: one possibility picked out of many.
The bent coin is the warning: a process that picks heads nine times in ten carries less than half a bit per flip — not the full bit a fair coin pays. Unfair processes are cheap to guess, and entropy is the price tag. Nothing in this chapter yet says who is guessing or how; that is Chapter 5's refinement.
One clarification before it causes harm, because it is the single most common confusion about the subject. Entropy is a property of the process that chose, never of the particular thing chosen. The twelve words apple apple apple… and the twelve words your wallet actually generated are, as strings, equally long and equally legal — but if the first came from a human picking a favorite fruit and the second from a fair roll of 2,048-sided dice, the first carries almost no entropy and the second carries 132 bits.
There is no machine that inspects a string and reports its entropy, the way there is no scale that weighs "how chosen" something was. Entropy lives in the how. Hold that thought: Chapter 9's graveyard is full of keys that looked perfectly respectable as strings.
Shannon's entropy is an average — the expected surprise over the whole process. Averages, as Chapter 3 kept warning, are not schedules, and they are not guarantees either. A process can have decent average surprise and still hand the thief a gift: one unusually likely outcome. The thief guesses the most likely value first, every time. So the security question is not "what is the average surprise" but "what does his best first guess buy?" That question has its own measure — leaner than Shannon's, harder-edged, and the one that actually prices your key. Chapter 5 builds it.
Exercise 4.1 ★ Corner the number. A number between 1 and 1,000,000 is chosen uniformly. How many halving questions guarantee you find it? Use Chapter 1's ladder. You should land on \(20\).
Exercise 4.2 ★ Bits on the card. How many bits of entropy does a uniformly chosen twelve-word phrase from the 2,048-word list carry? You should get exactly \(132\).
Exercise 4.3 ★★ The bent coin. Compute the entropy of a coin that lands heads 9 times in 10: entropy([0.9, 0.1]) using the chapter's two functions. Confirm 0.46899559358928133 — less than half a bit.
Exercise 4.4 ★★ Two loaded dice. Die \(A\) rolls its four faces with chances \((\tfrac{1}{2}, \tfrac{1}{4}, \tfrac{1}{8}, \tfrac{1}{8})\); die \(B\) with chances \((\tfrac{1}{2}, \tfrac{1}{2}, 0, 0)\) — it never shows 3 or 4 at all. Which has more entropy? Compute both: \(1.75\) bits and exactly \(1\) bit. Then state in one sentence what the difference measures.
Exercise 4.5 ★★★ The string tells you nothing. Your friend shows you a twelve-word phrase and asks, "does this have 132 bits of entropy?" Explain why the correct answer is you cannot tell from the string — what would you need to know instead? Anchor your answer in §4.5's distinction between the process and the product.
Sketches, not model answers — enough to check your reasoning against. The computational exercises embed their expected outputs in the exercise text.
Exercise 4.4. Die \(B\) has one bit: it is a coin in die's clothing, two equally likely faces. Die \(A\) has 1.75. The difference measures how much less predictable \(A\) is on average — \(A\)'s long shots (3 and 4, at 3 bits of surprise each) pull its average up even though both dice share the same most-likely face. Keep the two numbers in mind; the next chapter argues that for a thief, the shared \(\tfrac{1}{2}\) face matters more than the 0.75-bit gap.
Exercise 4.5. Entropy counts halvings in the choosing, and the string does not record its choosing. The same twelve words could come from a fair roll (132 bits) or from a human's favorites (a few bits at most, since favorite words cluster in a tiny corner of the phrase space). To answer the friend's question you need the process: what device, what generator, what source of physical noise. The string is the product; the entropy is the recipe. This is why the book's failure cases, when they come, are all stories about processes — never about strings that "looked random."