liveBook · Manning

NumPy array

As explained in Fast Python: High performance techniques for large datasets

A NumPy array, or ndarray, is a multidimensional array object with a defined shape and element data type, designed for efficient numerical computation.

A NumPy array holds multidimensional data for numerical processing. Its shape describes the dimensions; its dtype describes the elements. For a grayscale image, `np.array(image)` produces a two-dimensional array: a 182 × 45 image has shape `(45, 182)`—rows first, then columns—and `uint8` pixels. Check `shape`, `dtype`, and `nbytes` before changing how the data is processed. section 4.0 section 4.1.1

Inspect an image array

After loading a grayscale image with Pillow, run `image_arr = np.array(image)` and inspect `image_arr.shape`, `image_arr.dtype`, and `image_arr.nbytes`. For the 182 × 45 example, these report `(45, 182)`, `uint8`, and 8,190 bytes of array data. section 4.1.1

An operation need not allocate new pixel data. `np.flipud(image_arr)` can create a view: another interpretation of the same memory. `np.flipud(image_arr).copy()` instead owns a separate copy. Changing the original image then changes what the view displays, but not the copy. Use `np.shares_memory` to check which result shares storage; prefer a copy when later changes must remain independent. section 4.1.1

Table 4.1 Comparing the time and memory allocations between copies and views (view table figure)

Array size

Array memory (b)

View time

Copy time

1

2

0.171

0.281

10

20

0.137

0.259

100

200

0.139

0.286

1000

2000

0.162

0.502

10000

20000

0.142

2.275

100000

200000

0.138

31.257

200000

400000

0.152

67.005

400000

800000

0.144

354.287

800000

1600000

0.177

547.843

1000000

2000000

0.142

729.966

Compare view and copy timings as array size grows: view creation stays nearly flat in this benchmark, while copying becomes much more costly. section 4.1.1

Arrays also expose performance choices across libraries. A pandas series can yield its underlying array through `to_numpy()`; a vectorized tip calculation took 11 ms with NumPy arrays versus 35 ms with pandas objects in the reported run. Request `to_numpy(copy=True)` when changes must not affect the original, accepting the added time and memory cost. section 7.3.1

A typed NumPy array is not automatically cheap to index inside a Cython loop. Profiling an image operation found that `image[row, col]` still invoked Python machinery. For large arrays, disk representation matters too: Blosc wrote highly compressible zero and tiled arrays faster and smaller than NumPy’s uncompressed format in the reported tests, but was slightly slower for random data. Measure the operation and the data, not just the array API. section 5.3.2 section 6.2.1

How Fast Python: High performance techniques for large datasets covers it

How deep it goes
Works through shape, dtype, shared-memory views, and copy timings; defers internal memory organization to separate treatment.
Examples
Tests grayscale-image flipping and view timings, then stores zero, tiled, and random arrays with NumPy or Blosc.
What it assumes
Assumes some prior NumPy exposure while revisiting shape, dtype, and views from a performance perspective.
What you can do after
Apply it via NumPy and pandas to inspect arrays, choose views or copies, and benchmark vectorized calculations.
How it connects
Connects array views and vectorized arithmetic to pandas series, where copying changes memory use and timing.

Also explained in

Compare how 4 books explain NumPy array →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Fast Python: High performance techniques for large datasets. The section links point to the parts of the book it draws on.