1. The profiling funnel

Treat performance analysis as a sequence of gates. Do not move down a level until the current level identifies a question that the next tool can answer.

Figure 1.1 — Every gate reduces the search space and provides the selection criterion for the next profiler.

The optimization ceiling

Total runtime is the sum of every region's time, so no matter how much a chosen region is accelerated, everything outside it is an unmovable floor — this is why the chapter insists on estimating the ceiling before investing effort. If a region occupies fraction $p$ of runtime and is accelerated by $s$, Amdahl's law gives the end-to-end speedup

$$S = \frac{1}{(1-p)+p/s}.$$

Even deleting a 3% kernel completely yields only $1/(1-0.03)=1.031\times$. Use this calculation before spending an afternoon on a visually dramatic counter. The ceiling is unforgiving at small shares and only becomes worth chasing once a region dominates the total:

Decide which branch to follow

Figure 1.2 — Timeline continuity and family concentration decide whether the next tool should remain at system level or descend to counters.

An iteration with thousands of kernels but less than 1% device idle time is not currently launch-starved. Graph capture (CUDA Graphs, covered in ch. 3) may have solved dispatch even though the graph still replays thousands of internal nodes. Keep host API duration and actual GPU idle gaps conceptually separate.

Evidence hierarchy

Prefer evidence in this order:

1. repeatable end-to-end change;
2. timeline change that accounts for it;
3. counter change consistent with the timeline;
4. compiler or SASS change that explains the counter;
5. an intuition about what the hardware “should” do.

The last item is useful for forming hypotheses, not for declaring victory. Evidence closer to the top of this list is ranked higher because it is harder to fool yourself with: profiler overhead and measurement noise can produce an effect that looks real in a narrow, high-perturbation tool but disappears at the level that actually matters — unprofiled wall time.