Memory use is one signal to check when investigating slow code. High resource use can indicate a bottleneck or contention, so compare memory measurements with execution time and CPU and I/O activity. section 7.1.3
Distinguish the main limits before choosing what to optimize:
- CPU-bound tasks are limited by processing power; numerical simulations and sorting are examples. section 7.1.3
- I/O-bound tasks spend much of their time waiting on operations such as reading large files. Memory pressure can also lead to data being swapped to disk. section 7.1.3
def monitor_cpu_io_memory(duration, interval):
cpu_percentages = []
io_counters = []
memory_usages = []
end_time = time.time() + duration
while time.time() < end_time:
cpu_percentages.append(psutil.cpu_percent(interval=interval))
io_counters.append(psutil.disk_io_counters())
memory_usages.append(psutil.virtual_memory().used)
time.sleep(interval)
return cpu_percentages, io_counters, memory_usages
# Monitor CPU, I/O activity, and memory usage for 10 seconds with
1-second intervals
cpu_percentages, io_counters, memory_usages =
monitor_cpu_io_memory(duration=10, interval=1)
# Capture memory usage before running your_function()
memory_before = psutil.virtual_memory().used
def your_function():
# Your code here
pass
# Run your function
your_function()
# Capture memory usage after running your_function()
memory_after = psutil.virtual_memory().used
# Calculate function's memory usage
function_memory_usage = memory_after - memory_before
# Analyze CPU usage
avg_cpu_usage = sum(cpu_percentages) / len(cpu_percentages)
print(f"Average CPU usage: {avg_cpu_usage}%")
# Analyze I/O activity
read_bytes = [c.read_bytes for c in io_counters]
write_bytes = [c.write_bytes for c in io_counters]
print(f"Total read bytes: {sum(read_bytes)}")
print(f"Total write bytes: {sum(write_bytes)}")
# Analyze memory usage
avg_memory_usage = sum(memory_usages) / len(memory_usages) - memory_before
print(f"Average memory usage during function execution:
{avg_memory_usage / (1024 * 1024):.2f} MB")
print(f"Function memory usage:
{function_memory_usage / (1024 * 1024):.2f} MB")
A Python snippet samples CPU, disk I/O, and system memory over time, and compares memory before and after a function runs. section 7.1.3
