← Advanced course and all videos

Video 30 of 53 · 4:36 · Applications, benchmarking and evidence

The corrected B2/B4 comparison

In this video

  1. 0:00 · Introduction
  2. 0:16 · Read a corrected comparison without widening it
  3. 1:11 · Keep formula weighting distinct from row weighting
  4. 2:15 · Core explanation
  5. 3:33 · A wrapper can change the apparent advantage
  6. 4:26 · Direct takeaways

Caption transcript

Show transcript
This overview explains which scoped performance findings survived the corrected comparison with sharing-aware CSE-flat. Keeping kernel and wrapper results separate prevents a narrow retained advantage from becoming an unsupported universal claim. A comparison row is more than two elapsed times. It identifies the input, the implementation paths, the required output, and the boundary that was timed. Keep those fields attached to the corrected B2 and B4 result when interpreting it. A full vector at support sixteen contains sixty-five thousand five hundred thirty-six Boolean outputs. That tells us the assignment universe, not the complexity of every expression over those inputs. Two width-sixteen formulas can have very different structure. The measured comparison remains tied to its declared family, wrapper, and output task. It should not be turned into a universal width-sixteen statement or a point-query result. The corrected row is useful precisely when a new workload matches its scope. Formula balancing gives each formula equal influence despite different numbers of repeated timing rows. Row weighting gives every timing row equal influence. Consider a teaching example: formula X has ratio zero point five measured three times, and formula Y has ratio one point five measured once. For an arithmetic mean in this teaching example, the four rows average to zero point seven five. Giving each of the two formulas equal weight gives one. Repeating X changes the row-weighted result but not the equal-formula result. These teaching arithmetic means are not a reconstruction of the historical estimator. Formula-cluster uncertainty keeps each formula and its repeated rows together when resampling. It describes formula variation within the recorded run, not a guarantee across machines. The companion links the exact retained aggregation and ratio protocol; use that protocol for the historical numbers. The corrected row is useful precisely when a new workload matches its scope. The accepted B2/B4 V3 result compares compiled CM and sharing-aware CSE-flat after compilation. The formula-balanced CM-over-CSE-flat time ratio is 0.8906, with a formula-cluster 95% interval from 0.8741 to 0.9073. At live support 16, that ratio narrows to 0.9612, with interval 0.9290 to 0.9942. These are two labelled cohort estimates. Neither is a speedup that applies to every formula. In the same correction history, the public wrapper is slower than CSE-flat; the bare evaluator result does not pay for all request stages. These are historical retained estimates for their recorded workload, host, comparator and timing boundary. They are not a benchmark of today's modified code. The cluster intervals describe within-run formula variation, not an all-machine guarantee. Current implementation changes require a separately authorized matched replay before new performance claims. Consider fictional costs that demonstrate the logic of a correction. A six-unit candidate compared with a ten-unit control has a time ratio of zero point six. These teaching numbers are not the B2 or B4 measurements. If a corrected wrapper reduces the control to five units while preserving the same output, the candidate's new ratio is one point two. The candidate did not become logically wrong. The old performance comparison stopped representing the stronger baseline. Preserve the historical record and clearly identify the correction. Claims that depended on the weaker comparison must be revised. Other claims require their own evidence; they neither disappear automatically nor gain support from the correction. Preserve the retained result with its workload, host, and timing boundary. New implementation claims require a new matched comparison.