line_profiler measures time spent on individual source-code lines. Use it when a function-level profile has identified a slow function but cannot show which statements inside it are costly. For example, aggregate time in math.sin cannot distinguish between several lines that call it. section 2.2.1 section 2.2.2
Profile a Python function in two passes:
- Run cProfile first to locate a slow function. It is faster than line profiling and reports function-level time and subcalls. section 2.2.3
- Add @profile to the target function. Run `kernprof -l lprofile_distance_cache.py`; kernprof supplies the decorator, so it need not be imported in the script. section 2.2.2
- Inspect the saved trace with `python -m line_profiler lprofile_distance_cache.py.lprof`. Read hits, total time, time per hit, and percentage of function time alongside each source line. section 2.2.2
Finding the expensive statement
A distance calculation calls trigonometric functions on multiple lines. Function-level profiling identifies get_distance as costly but cannot attribute a shared math function’s cost to one call site. Line profiling breaks down get_distance statement by statement, directing attention to lines worth optimizing. section 2.2.1 section 2.2.2
Instrumentation can slow execution by several orders of magnitude. A long run can be interrupted and still leave a trace. Treat the results primarily as a way to locate bottlenecks, not as timings directly comparable with an uninstrumented run. section 2.2.2 section 5.3.2
Cython functions can also be line-profiled, but they need tracing enabled in the Cython code and `CYTHON_TRACE` defined when compiling. The caller can create a `line_profiler.LineProfiler`, run the function with `runcall`, and print its statistics. In an image-filtering example, this exposed a pixel assignment consuming 60% of the profiled time. section 5.3.2
import pyximport
import line_profiler #1
import numpy as np
from PIL import Image
pyximport.install(
language_level=3,
setup_args={
'options': {"build_ext":
{"define": 'CYTHON_TRACE'}}, #2
'include_dirs': np.get_include()})
import cyfilter_lprof as cyfilter
image = Image.open("../../04-numpy/aurora.jpg")
gray_filter = Image.open("../filter.png").convert("L")
image_arr, gray_arr = np.array(image), np.array(gray_filter)
profile = line_profiler.LineProfiler(
cyfilter.darken_annotated) #3
profile.runcall(cyfilter.darken_annotated, image_arr, gray_arr)
profile.print_stats()
The caller shows the Cython-specific setup: define CYTHON_TRACE through pyximport, profile the compiled function with LineProfiler.runcall, and print line statistics. section 5.3.2
