cProfile is Python’s built-in deterministic profiler for locating bottlenecks before optimization. It records function-call counts and timing statistics without requiring changes to the profiled program. section 2.1.2
Use the profiler output to interpret where execution time goes:
- Sort results by cumulative time to find functions whose own work and subcalls consume the most time. section 2.1.2
- Read ncalls for invocation frequency, tottime for time excluding subcalls, and cumtime for time including subcalls. section 2.1.2
- Treat network and disk I/O as possible dominant costs, especially when timings vary between runs. section 2.1.2
- Escalate to line profiling when function-level cumulative values cannot explain the bottleneck; cProfile is faster but less detailed. section 2.2.3
Profile a data-loading script
Run `python -m cProfile -s cumulative load.py ... > profile.txt`, then inspect the report. A data-loading run shows `download_all_data` dominating cumulative time, with network calls and socket reads accounting for much of that cost.
python -m cProfile -s cumulative load.py 01044099999,02293099999 2021-2021 > profile.txt
Use the runnable command to profile a script and save results ordered by cumulative time. section 2.1.2
