WEBVTT

1
00:00:00.200 --> 00:00:05.364
When feedback is noisy, a surprising report need not mean the world has changed.

2
00:00:05.364 --> 00:00:10.507
Reliable prediction requires an estimate that can respond to evidence without treating

3
00:00:10.507 --> 00:00:13.227
every report as a new physical event.

4
00:00:13.227 --> 00:00:18.733
Watch the distinction between changing a state, updating its estimate and revising the

5
00:00:18.733 --> 00:00:19.720
model itself.

6
00:00:21.040 --> 00:00:25.780
Keep the same PULSE and RESET rules, with no spontaneous changes.

7
00:00:25.780 --> 00:00:28.800
Replace the exact reader with a noisy one.

8
00:00:28.800 --> 00:00:32.320
We label this noisy-reader variation M1.

9
00:00:32.320 --> 00:00:41.640
In this teaching variation, it reports the true lamp value with probability eighty percent and the opposite value with probability twenty percent.

10
00:00:41.640 --> 00:00:44.336
Reads still leave the device unchanged.

11
00:00:44.336 --> 00:00:55.140
When the hidden configuration is held fixed, the report errors are independent: an earlier error does not change the probabilities of the next report for that fixed configuration.

12
00:00:55.140 --> 00:01:02.160
The transition rules and sensor probabilities are supplied assumptions; we have not learned them from the reports.

13
00:01:02.950 --> 00:01:07.782
Begin a new run with half the starting probability on u and half on v.

14
00:01:07.782 --> 00:01:13.270
This equal prior is a new assumption, not something implied by the two state names.

15
00:01:13.270 --> 00:01:16.740
Prior means the estimate before the next piece of evidence.

16
00:01:16.740 --> 00:01:18.190
Execute PULSE.

17
00:01:18.190 --> 00:01:23.624
The half assigned to u moves to w; the half assigned to v stays at v.

18
00:01:23.624 --> 00:01:29.563
Before any new report, the predicted current probabilities are half w and half v.

19
00:01:29.563 --> 00:01:31.750
No probability is lost.

20
00:01:32.540 --> 00:01:34.140
The reader reports one.

21
00:01:34.140 --> 00:01:41.447
If the current state is w, that report has probability eighty percent, because w has lamp one.

22
00:01:41.447 --> 00:01:48.100
If it is v, the report has probability twenty percent, because the reader can misreport its zero lamp.

23
00:01:48.100 --> 00:01:51.868
These conditional report probabilities are likelihoods.

24
00:01:51.868 --> 00:01:57.148
Keep the one actual received report separate from the two possible explanations.

25
00:01:57.148 --> 00:02:02.060
Both w and v remain possible, with different support from this evidence.

26
00:02:02.850 --> 00:02:13.020
To make the update visible, scale the starting probability to one hundred equal units of model weight: fifty for w and fifty for v.

27
00:02:13.020 --> 00:02:18.880
These are drawing units for probabilities, not a hundred observed devices or experiments.

28
00:02:18.880 --> 00:02:25.270
For the w case, eighty percent of its fifty weight units correspond to report one.

29
00:02:25.270 --> 00:02:30.224
That gives forty units of weight for w together with this report.

30
00:02:30.224 --> 00:02:37.204
For v, twenty percent of its fifty units correspond to report one, giving ten units.

31
00:02:37.204 --> 00:02:43.694
The remaining model weight belongs to report zero, which is not the report received in this step.

32
00:02:43.694 --> 00:02:51.174
Among the weight matching report one, there are forty units for w and ten for v, fifty in total.

33
00:02:51.174 --> 00:02:55.554
The w share is forty divided by fifty: eighty percent.

34
00:02:55.554 --> 00:02:58.574
The v share is twenty percent.

35
00:02:58.574 --> 00:03:01.521
These are the updated state probabilities.

36
00:03:01.521 --> 00:03:08.814
Is the state now certain to be w?

37
00:03:08.814 --> 00:03:11.274
No. Both states remain possible.

38
00:03:11.274 --> 00:03:14.014
Now suppose the prior had been different.

39
00:03:14.014 --> 00:03:23.174
Would the same reader and report always give eighty percent on w?

40
00:03:23.174 --> 00:03:29.294
No. The eighty-percent result used our half-and-half prior as well as the specified sensor.

41
00:03:29.294 --> 00:03:34.534
A different prior can give a different updated probability with the same reader and report.

42
00:03:35.244 --> 00:03:39.904
Take another reading, with no action or state change between readings.

43
00:03:39.904 --> 00:03:45.297
Use the same sensor rule with a fresh error draw for that unchanged state.

44
00:03:45.297 --> 00:03:46.764
It reports zero.

45
00:03:46.764 --> 00:03:55.014
Use the previous updated estimate as the new prior: eighty percent on w and twenty percent on v.

46
00:03:55.014 --> 00:04:02.624
On the same one-hundred-unit scale, w contributes eighty times twenty percent, or sixteen units.

47
00:04:02.624 --> 00:04:08.504
The v case contributes twenty times eighty percent, also sixteen units.

48
00:04:08.504 --> 00:04:13.514
For this second update, the total matching weight is thirty-two units.

49
00:04:13.514 --> 00:04:21.108
Each state gets sixteen of those thirty-two units, so the updated probabilities return to one half each.

50
00:04:21.108 --> 00:04:25.508
Our model allowed different reports without a physical change.

51
00:04:25.508 --> 00:04:29.408
The disagreement alone therefore does not prove the device changed.

52
00:04:30.198 --> 00:04:36.385
A controller can predict, receive a report, update its estimate and choose its next operation.

53
00:04:36.385 --> 00:04:41.158
If its goal is to report the hidden current condition, another reading might help.

54
00:04:41.158 --> 00:04:45.665
Whether it is worth taking depends on the decision rule and its costs.

55
00:04:45.665 --> 00:04:53.974
If the goal is instead to turn the lamp on, RESET followed by PULSE succeeds under our known rules, even without feedback.

56
00:04:53.974 --> 00:04:58.078
That changes the device; it does not reveal its origin.

57
00:04:58.078 --> 00:05:03.158
A loop diagram by itself does not establish a benefit over alternative plans.

58
00:05:03.948 --> 00:05:10.768
Our worked calculation estimated the current state using a supplied transition rule and sensor model.

59
00:05:10.768 --> 00:05:15.184
Learning a previously unknown response rule is a different task.

60
00:05:15.184 --> 00:05:19.100
Finish the operating example before considering that new question.

61
00:05:19.100 --> 00:05:25.590
Imagine extra calibration equipment that can prepare a known v and record responses to PULSE.

62
00:05:25.590 --> 00:05:29.134
Those trials could help compare candidate response rules.

63
00:05:29.134 --> 00:05:36.210
This preparation access was not available to the operating observer, and RESET does not prepare v.

64
00:05:36.210 --> 00:05:38.477
No such experiment has been run here.

65
00:05:39.347 --> 00:05:45.164
A future investigation might find that prediction requires an unrecorded mode or recent history.

66
00:05:45.164 --> 00:05:49.400
Adding such a variable would revise the model’s state description.

67
00:05:49.400 --> 00:05:56.174
It would be a new model to justify and test, not a hidden change to the rules of our existing example.

68
00:05:56.174 --> 00:06:03.605
Noise, an incorrect response rule and an omitted variable are different possible explanations for discrepancies.

69
00:06:03.605 --> 00:06:05.703
Evidence must help compare them.

70
00:06:05.703 --> 00:06:12.707
Updating the state estimate, learning a rule and revising the variables should not be treated as the same operation.

71
00:06:13.497 --> 00:06:27.204
Classify these operations: change the probabilities on w and v; estimate a previously unknown PULSE response rule; add memory of the previous output to the description.

72
00:06:27.204 --> 00:06:37.057
Which is state estimation, rule learning and model revision?

73
00:06:37.057 --> 00:06:40.097
Changing the probabilities is state estimation.

74
00:06:40.097 --> 00:06:43.000
Estimating the response rule is rule learning.

75
00:06:43.000 --> 00:06:46.097
Adding memory revises the state description.

76
00:06:46.097 --> 00:06:50.857
None of those descriptions alone says that a physical action was executed.

77
00:06:50.857 --> 00:06:56.287
Episode eight introduces a separately specified threshold-and-memory example.

78
00:06:56.287 --> 00:07:04.004
Episode nine then changes to the R4 actuator case, with its own dynamics, probabilities and evidence.

79
00:07:04.004 --> 00:07:09.137
Carry the distinctions forward, and introduce each new model on its own terms.