liveBook · Manning

Array programming

As explained in Fast Python: High performance techniques for large datasets

Array programming is a programming technique that expresses operations over whole arrays rather than iterating imperatively over individual elements.

Array programming applies operations to whole arrays instead of iterating over their elements in Python. With NumPy, adding two vectors can be written as `a + b` rather than a loop that reads, adds, and stores each pair of values. The shorter expression states the intended operation directly. section 4.2

Using an array library does not automatically mean using array programming: NumPy arrays can still be processed element by element in Python. Whole-array operations can be much faster because the operation runs in an implementation typically written in C or Fortran, rather than repeatedly executing Python loop code. Broadcasting extends this style to operations involving arrays of different dimensions. The speed advantage is a reason to look for array expressions, not a substitute for measuring a particular workload. section 4.2 section 4.2.1 section 4.4

import numpy as np
 
def sum_arrays(a, b):  # Assumes both are the same size
    my_sum = np.empty(a.size, dtype=a.dtype)
    for i, (a1, b1) in enumerate(zip(np.nditer(a), np.nditer(b))):
        my_sum[i] = a1 + b1
    return my_sum.reshape(a.shape)

The element-by-element baseline uses NumPy arrays but still performs the additions in a Python loop. section 4.2

a + b

The whole-array version expresses the same intended vector addition with one operator. section 4.2

Brightening an image without silent overflow

An image stored as one-byte unsigned integers can be brightened with `image_arr + 5` or `image_arr * 2`, but either operation can overflow the 0–255 pixel range. Adding 5 to 255 produces 4; doubling can yield an image that looks plausible while its maximum is only 254. One corrective approach is to convert to `np.uint16`, perform the operation, limit values with `np.minimum(..., 255)`, and convert back to `np.uint8`. Whole-array code is concise, but its data type and boundary values still need testing. section 4.2.3

How Fast Python: High performance techniques for large datasets covers it

How deep it goes
Defines the model, contrasts two vector-addition implementations, and works through image overflow; stops short of benchmarking those examples.
Examples
Contrasts loop-based and whole-array vector addition, then uses image brightening to expose unsigned-integer overflow.
What it assumes
Assumes some NumPy exposure but distinguishes owning an array from programming with whole-array operations.
What you can do after
Apply it via NumPy to replace suitable loops while checking data types and boundary values.
How it connects
Connects whole-array operations to NumPy, broadcasting, data types, and compiled implementations.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Fast Python: High performance techniques for large datasets. The section links point to the parts of the book it draws on.