Hardware is the physical computing equipment that runs software. For performance work, it includes the CPU, memory, local storage, and network. An optimization that helps on one architecture may be a poor choice on another, so the available resources matter alongside the data and algorithm. section 6.1 section 1.2
Follow where data lives and moves: through CPU caches, RAM, disks, local networks, and wider networks. These layers affect both processing speed and storage efficiency. For example, working with compressed data can sometimes be faster than working with uncompressed data despite the extra decompression, overturning a simple computation-only cost estimate. section 6.0
Account for a GPU transfer
To multiply every element of a 1024 × 64 matrix by 2 on a GPU, first transfer the matrix from CPU-side memory to GPU memory. The 65,536 elements can then run as separate threads. At an assumed 32 threads per block, that is 2,048 blocks. Transfer cost may outweigh the computation when the operation is this small; the CPU launches the GPU kernel. section 9.1.4
Choose an approach for the machine and workload rather than applying every available optimization. CPU design, GPUs, storage, networks, clusters, and cloud environments offer different advantages and trade-offs; Python libraries can help exploit particular hardware effects. section 1.4 section 6.1
