Being and Becoming · Episode 4 of 10 · 5:05

What information matters for the task?

Useful for which target, prediction or decision?

Keyboard: focus the player, then Space or K to play/pause and arrow keys to seek. Tab moves through the controls. Selectable captions replace the video's caption strip with the same timed text below.

Open MP4 English captions (VTT)Read transcript Download transcript

In this episode

Complete transcript

This transcript includes the answers to prediction prompts.

Show transcript

Why this question matters · 0:00

More data is not automatically more useful. Its value depends on what we are trying to predict or decide. We therefore need to separate how much remains unknown from whether that uncertainty matters for the task. We choose facts useful for the task while the card's Being stays fixed. Becoming would require a change in that state. Watch which distinctions the task needs.

Declare the task and reset the example · 0:26

Episode three kept the distinctions needed for a prediction. To measure uncertainty, we need more than a list of possibilities: we need probabilities and a target. Return to the four-card example. The short existing overview names four meanings of information; then we will work one of its calculations step by step.

Existing overview: four meanings of information · 0:46

The word information can point to several different things. Here are four possible states. A is the first digit; B is the second. Suppose an exact readout says A is zero. That answer is an observation record. Now ask what this test can distinguish. Reading A separates states starting with zero from states starting with one. It leaves the second digit unresolved. This is a pattern of distinctions, not yet an uncertainty score. To measure uncertainty, supply probabilities and name a target. If the four states are equally likely, reading A leaves one bit of uncertainty about the full state. That number summarizes the probability model; it does not list the missing distinctions. Finally, name the task. If we need A, this readout gives the answer. Change the target to B, and it gives no advantage under our independent, fair model. Record, distinctions, uncertainty, and task value are related questions. Keeping them separate helps us say exactly what changed when we gained information.

Supply the uniform distribution · 1:50

Assign each card probability one quarter. Let X name the whole card. One fair zero-or-one choice has one bit of uncertainty. This card combines two independent fair choices, so its Shannon entropy is two bits. Entropy averages uncertainty under a specified probability distribution. It differs from the number of digits used to write the answer. We will use this fair case first; the optional entropy primer also works a biased example.

Condition on an answer, then average · 2:22

Read A and suppose it is zero. Zero-zero and zero-one survive. Their original probabilities were one quarter each; the accepted event had total probability one half. Within it, each card therefore has conditional probability one half. One fair choice remains, so one bit of full-card uncertainty remains. The A-equals-one branch also leaves one bit. Conditional entropy averages these remaining uncertainties over the possible answers; here the average is one bit.

Change the target to shape · 2:58

Now the target is B alone. Reading A still leaves B equally likely to be zero or one, so one bit about B remains. Reading B itself leaves no uncertainty about B. The average reduction in uncertainty about a named target is called mutual information. In this distribution, A supplies zero bits about B, while B supplies one. These numbers depend on the target and the model.

Equal entropy can support different decisions · 3:27

Suppose the task is to predict B correctly, scoring one for a correct answer and zero otherwise. Averaged over the four equally likely cards, the best accuracy using only A is fifty percent. Reading B gives one hundred percent accuracy under this exact model. Yet either single test leaves one bit about the whole card. Equal full-state residual entropy does not imply equal task value. Change the task to A, and which reader is useful reverses.

Keep the distribution and averaging visible · 4:02

The four probabilities were assumptions, not consequences of there being four cards. Changing them can change the numerical entropies and decision scores. Also, conditional entropy is an average across answers; a particular rare answer can leave more uncertainty than before. The optional foundations explain that case. Our core calculations used a fair distribution and exact observations, not a general rule that every question removes one bit.

Choose information for a purpose · 4:32

For a final check, the task is shape prediction and you may read one of the two card properties. Which do you choose? B, because it directly resolves the target under this model. To compare richer strategies, we must also name available tests, permitted timing and costs. That is the next episode’s job. Later, the noisy-reader lesson will announce its own prior instead of borrowing these four equal card probabilities.