Profiling measures where a program spends time so optimization starts with evidence rather than a guess. A slow data pipeline may be waiting on network I/O, reading files, or doing computation; the most promising change depends on which cost dominates. section 2.0 section 2.1.2
Use progressively finer measurements:
- Run `python -m cProfile -s cumulative script.py` first. Read call counts, time in a function, and cumulative time including its subcalls. A function with high cumulative time may only be waiting on functions it calls. section 2.1.2
- Save a trace with `python -m cProfile -o run.prof script.py`, then inspect its call structure with `snakeviz run.prof` or Python’s `pstats` module. section 2.2.1
- If a costly function remains ambiguous, use line profiling. It locates expense within the function, but collecting that detail is substantially slower than built-in function profiling. section 2.2.3
- Use `%timeit` to compare small operations over repeated runs, not as a substitute for locating costs across a whole program. section 2.2.3
A slow climate-data loader
A loader downloads NOAA station records, writes CSV files, reads temperatures, and computes each station’s minimum. Its cProfile run takes 3.061 seconds; downloading has 2.768 seconds of cumulative time, while the minimum-temperature function registers effectively none. The download path includes network reads, so optimizing the final calculation would target the wrong part of this run. Network timings can vary between runs. section 2.1.1 section 2.1.2
Choose measurements that match the resource. For data loading, test the actual files and conditions before trading read time against compressed size. For GPU work, ordinary CPU-side `%timeit` does not represent GPU execution cost: CuPy’s `cupyx.time.repeat` reports CPU and GPU times separately, while NVIDIA Nsight Systems can expose transfers and kernel activity. section 7.1.1 section 9.3.3 section 9.3.6
Time% Total ns Calls Avg ns Min ns Max ns StdDev Name ------ --------- ----- --------- --------- ---------- ------ -------------- 96.1 368748545 1 368748545 368748545 368748545 0 cuMemcpyDtoH 3.6 13654495 1 13654495 13654495 13654495 0 cuMemcpyHtoD 0.1 540957 2 270478 234726 306231 50561 cuMemAlloc 0.1 371672 1 371672 371672 371672 0 cuModLdDataEx 0.0 133176 1 133176 133176 133176 0 cuLinkComplete 0.0 66248 1 66248 66248 66248 0 cuLinkCreate 0.0 49602 1 49602 49602 49602 0 cuMemGetInf 0.0 37495 1 37495 37495 37495 0 cuLaunchKernel 0.0 2071 1 2071 2071 2071 0 cuLinkDestroy
In this GPU trace, device-to-host and host-to-device copies account for nearly all reported GPU-call time. The expensive part is moving data, not launching the kernel. section 9.3.6
