NumPy provides multidimensional arrays and operations on them. Its C- and Fortran-based core makes it a practical foundation for numerical work in Python, including data-processing libraries such as pandas. An image, for example, can be treated as a two-dimensional array; the performance question is how to manipulate it without unnecessary work or allocation. section 4.0 section 4.1
Three choices have a large effect on array workloads:
- Prefer views over copies when sharing the original data is acceptable. A view changes the perspective on existing data with little computing or memory cost; its shape gives dimension sizes, while its strides give byte steps between elements. section 4.4
- Express work as operations on whole arrays rather than element-by-element loops. Broadcasting supports operations across arrays of different dimensions. section 4.4
- Check the numerical libraries behind the installation. BLAS and LAPACK choices affect performance, and their threading behavior matters before adding Python multiprocessing around NumPy calls. section 4.3.2 section 4.4
Moving a pandas calculation onto arrays
Extract columns with `to_numpy()`, then perform the calculation on the resulting arrays. In one measured tip-analysis workload, vectorized NumPy code took 11 ms versus 35 ms for vectorized pandas code on the same computer. The default conversion can share underlying data with the pandas object; `to_numpy(copy=True)` avoids changes reaching the original but adds copying time and memory use. Benchmark the actual workload rather than assuming the same gain elsewhere. section 7.3.1 section 7.3.3
When array operations are not enough, NumPy arrays can also supply data to Cython code. That route, and alternatives such as NumExpr, should follow measurement: their relative results depend on the algorithm, data, software and hardware. section 7.3.3 section 5.1
