WEBVTT

1
00:00:00.000 --> 00:00:08.333
This overview explains how ratio direction and uncertainty
affect the interpretation of a timing comparison.

2
00:00:08.333 --> 00:00:17.191
Reversing numerator and denominator reverses which side
of one looks favorable without changing the underlying times.

3
00:00:17.191 --> 00:00:17.200
Reversing numerator and denominator reverses which side
of one looks favorable without changing the underlying times.

4
00:00:17.500 --> 00:00:22.367
Declare the ratio as candidate time divided by control time.

5
00:00:22.367 --> 00:00:28.333
With illustrative times eight and ten milliseconds, the ratio is eight divided by ten, or

6
00:00:28.333 --> 00:00:30.100
zero point eight.

7
00:00:30.100 --> 00:00:35.467
A smaller time ratio favors the candidate under this convention.

8
00:00:35.467 --> 00:00:39.867
The candidate saves two milliseconds out of the control's ten.

9
00:00:39.867 --> 00:00:43.300
That is twenty percent less elapsed time.

10
00:00:43.300 --> 00:00:49.033
This percentage uses the control time as its denominator, so it is not the same number as

11
00:00:49.033 --> 00:00:51.967
the throughput increase.

12
00:00:51.967 --> 00:00:56.400
The reciprocal time ratio is one point two five.

13
00:00:56.400 --> 00:01:02.333
Under comparable repeated work, that corresponds to twenty-five percent higher throughput.

14
00:01:02.333 --> 00:01:07.967
Keep the workload and concurrency assumptions explicit; do not swap percentages without

15
00:01:07.967 --> 00:01:11.466
changing their denominators.

16
00:01:11.466 --> 00:01:11.467
changing their denominators.

17
00:01:11.467 --> 00:01:15.000
First read the numerator and denominator.

18
00:01:15.000 --> 00:01:21.167
For a time ratio M over C, a value below one means M took less time.

19
00:01:21.167 --> 00:01:25.833
Reverse the ratio and the same observation appears above one.

20
00:01:25.833 --> 00:01:29.667
Use toy times of 8 and 10 milliseconds.

21
00:01:29.667 --> 00:01:35.233
M over C is 0.8; C over M is 1.25.

22
00:01:35.233 --> 00:01:41.300
That is 20% less time for M, or a 1.25 times reciprocal speedup.

23
00:01:41.300 --> 00:01:44.300
The parity line stays at one.

24
00:01:44.300 --> 00:01:49.767
Add an uncertainty interval only when its
procedure and sampling unit are known.

25
00:01:49.767 --> 00:01:55.767
An interval spanning one does not establish
a directional advantage at that interval level.

26
00:01:55.767 --> 00:02:02.433
A formula-cluster interval from one run
does not estimate variation over all hosts.

27
00:02:02.433 --> 00:02:09.900
Also, a larger throughput ratio has a different
favourable direction from a smaller elapsed-time ratio.

28
00:02:10.200 --> 00:02:15.400
An estimated ratio close to one may come with an interval crossing one.

29
00:02:15.400 --> 00:02:19.500
Under the method used to construct that interval, both sides of parity

30
00:02:19.500 --> 00:02:22.200
remain within the reported range.

31
00:02:22.200 --> 00:02:28.067
The point estimate alone does not support a confident strict-win statement.

32
00:02:28.067 --> 00:02:33.000
An interval entirely below one supports a different statement about this comparison under

33
00:02:33.000 --> 00:02:34.767
its sampling method.

34
00:02:34.767 --> 00:02:39.967
It still does not demonstrate the same result on every machine, every input family, or

35
00:02:39.967 --> 00:02:42.733
every output task.

36
00:02:42.733 --> 00:02:46.500
Repeated timing samples, new formulas, and new hosts address

37
00:02:46.500 --> 00:02:48.967
different sources of variation.

38
00:02:48.967 --> 00:02:53.133
State which unit was sampled and which interval was calculated.

39
00:02:53.133 --> 00:03:00.359
Three host values are not automatically a confidence interval for a larger population.

40
00:03:00.359 --> 00:03:00.367
Three host values are not automatically a confidence interval for a larger population.

41
00:03:00.367 --> 00:03:05.200
Read the numerator, denominator, and sampling unit first.

42
00:03:05.200 --> 00:03:11.933
An interval crossing one does not establish
a directional advantage at that interval level.