5. Select kernels by impact
Nsight Compute can replay one launch dozens of times. Profile a small,
representative set rather than attaching --set full to an entire application.
Rank by contribution, then by distinct behavior
For each kernel family, compute
Select the largest contributors, then sample separate resource signatures or shapes within a family. A GEMM family with four shapes may need four samples; 800 identical elementwise launches usually need one.
| Selection criterion | Why it matters |
|---|---|
| high total time | end-to-end ceiling |
| long individual duration | stable counter sampling |
| high launch count | fusion/dispatch opportunity |
| unusual register/shared-memory signature | different occupancy regime (ch. 7) |
| different shape or tail wave (grid wave efficiency, ch. 10) | different tile efficiency |
| regression between revisions | direct causal target |
Preserve the production context
Ranking picks which kernel to profile; it says nothing about which invocation of that kernel is representative. The same symbol can behave differently with different shapes, alignment, cache state, concurrent streams, clocks, or data distributions. Prefer capturing a production launch with a kernel-name, NVTX, or launch-index filter. Use a standalone reproducer only after verifying its launch metadata and duration match the original.
Estimate the upper bound first
Before spending engineering time on a captured kernel, bound what fixing it can possibly buy. If family share is , a miraculous 2x kernel speedup produces only
That kernel may still be worth fixing if the same defect appears across related kernels, but the argument must be aggregate.
Avoid these selection traps
Even a disciplined ranking process fails in predictable ways:
- Profiling the first matching launch, which is often initialization or a small shape.
- Choosing the kernel with the lowest occupancy instead of the largest impact.
- Combining forward, backward, and recompute launches under one average.
- Comparing raw replay durations collected at substantially different clocks.
- Treating a library kernel's long C++ template name as a unique workload.
The nsys trace is the index into ncu: record the base symbol, occurrence,
grid, block, registers/thread, dynamic shared memory, stream, and enclosing
range before collecting counters.