12. Controlled performance experiments

A counter report should end in an experiment, not an adjective.

Write the hypothesis first

Use this form:

Because evidence, changing one mechanism should improve specific counters and reduce specific elapsed time, without changing correctness constraints.

Example:

Because a backward kernel is register-limited to eight active warps/SM and local-memory loads dominate its long-scoreboard samples, shortening the live range of the per-thread accumulator should reduce local sectors and no-eligible cycles, then reduce both kernel and iteration time.

Keep an experiment ledger

Turn each hypothesis into a row before running it, and keep every prior row instead of overwriting it — the ledger is what lets a later result be compared against the reasoning that motivated it:

Revision	Workload	Change	Median	p20–p80	Key counters	Correct?
baseline	M,N,K / batch,seq	none	…	…	…	yes
A	same	smaller tile	…	…	regs, spills, Tensor%	yes/no

Do not combine tile size, precision, fusion, and launch geometry in one patch if you want to understand causality. Change several of them at once and a favorable result cannot be attributed to any one of them — an improvement without a known mechanism is a lucky patch, not the kind of causal, generalizable understanding of the kernel that lets you predict its behavior on the next shape or the next change.

A favorable median in the ledger is not yet a validated result — it still has to survive the same causal chain the hypothesis predicted, link by link.

Optimization proof chain

Figure 12.1 — A broken link is a result to explain, not permission to skip directly to the end-to-end claim.

If the chain breaks:

generated code unchanged: the compiler optimized away or restructured the edit;
counters improve but kernel does not: another resource became limiting;
kernel improves but pass does not: launch is too small or another shape dominates;
pass improves but end-to-end does not: overlap or an outside bottleneck hides it;
end-to-end improves only under profiler: measurement perturbation or noise.

A chain that holds end to end proves the change is fast. It says nothing about whether the change is still right, which is a separate and equally required check.

Correctness is multidimensional

For numerical kernels check representative distributions, edge shapes, non-contiguous layouts, tails, NaN/Inf behavior, deterministic requirements, and backward gradients where applicable. A faster reduced-precision path that changes convergence is an algorithm change, not a free kernel optimization.

Re-profile at the right level