Video 33 of 53 · 3:12 · Applications, benchmarking and evidence
Reading timing ratios and intervals
In this video
Caption transcript
Show transcript
This overview explains how ratio direction and uncertainty affect the interpretation of a timing comparison.
Reversing numerator and denominator reverses which side of one looks favorable without changing the underlying times.
Declare the ratio as candidate time divided by control time.
With illustrative times eight and ten milliseconds, the ratio is eight divided by ten, or
zero point eight.
A smaller time ratio favors the candidate under this convention.
The candidate saves two milliseconds out of the control's ten.
That is twenty percent less elapsed time.
This percentage uses the control time as its denominator, so it is not the same number as
the throughput increase.
The reciprocal time ratio is one point two five.
Under comparable repeated work, that corresponds to twenty-five percent higher throughput.
Keep the workload and concurrency assumptions explicit; do not swap percentages without
changing their denominators.
First read the numerator and denominator.
For a time ratio M over C, a value below one means M took less time.
Reverse the ratio and the same observation appears above one.
Use toy times of 8 and 10 milliseconds.
M over C is 0.8; C over M is 1.25.
That is 20% less time for M, or a 1.25 times reciprocal speedup.
The parity line stays at one.
Add an uncertainty interval only when its procedure and sampling unit are known.
An interval spanning one does not establish a directional advantage at that interval level.
A formula-cluster interval from one run does not estimate variation over all hosts.
Also, a larger throughput ratio has a different favourable direction from a smaller elapsed-time ratio.
An estimated ratio close to one may come with an interval crossing one.
Under the method used to construct that interval, both sides of parity
remain within the reported range.
The point estimate alone does not support a confident strict-win statement.
An interval entirely below one supports a different statement about this comparison under
its sampling method.
It still does not demonstrate the same result on every machine, every input family, or
every output task.
Repeated timing samples, new formulas, and new hosts address
different sources of variation.
State which unit was sampled and which interval was calculated.
Three host values are not automatically a confidence interval for a larger population.
Read the numerator, denominator, and sampling unit first.
An interval crossing one does not establish a directional advantage at that interval level.

