A NumPy array is a multidimensional, in-memory data structure common in Python’s scientific computing ecosystem. Its core properties are data, shape, and dtype. section 8.1.1
Inspect and prepare an array through its structure and values:
- Represent MNIST training images as a 60,000 × 28 × 28 array of uint8 values ranging from 0 to 255. section 8.1.1
- Inspect data, shape, dtype, minimum, and maximum values before modeling. section 8.1.1
- Preprocess image features by dividing values by 255, producing values from 0.0 to 1.0 and a float64 array. section 8.1.1
Keeping a large NumPy array in memory can become a scalability problem. Feeding it to TensorFlow through tf.constant() may copy its contents multiple times; a 1.5-GB array can exceed the 2-GB tf.GraphDef protocol-buffer limit after two copies. Additional transformations can require still more memory. section 2.3.1
MNIST training pipeline
Load MNIST through tf.keras.datasets.mnist.load_data(), receive NumPy arrays for training and testing, scale the image arrays, and pass them to a Keras model for training and evaluation. section 8.1.1
Figure 2.5 An example 1.5-GB in-memory NumPy array representation that hits an out-of-memory error when being converted to a tf.GraphDef protocol buffer
Visualizes how copying a 1.5-GB in-memory NumPy array into a TensorFlow graph can produce an out-of-memory failure. section 2.3.1
