Chapter 4: Real-world data representation using tensors

Introduction

This introduction explains why representing real-world data as tensors matters for using PyTorch neural networks.

• Tensors throughout neural networks: Explain why data must be represented as tensors for a PyTorch neural network: networks accept and produce tensors, and their operations and parameters are tensors too.
  Why it matters: This is why turning real-world data into tensors is a necessary step before using it with a PyTorch model.

Key terms: tensors, inputs, outputs, parameters, weights, biases

4.1 Working with images

This section introduces images as grids of numerical pixel values and explains why those values must be represented appropriately as PyTorch tensors.

• Image grids and pixel values: Explain that an image is a grid of pixel values in rows and columns, with either one value per pixel for grayscale or multiple values for color or other features. State that using images with PyTorch requires loading them and arranging their data in a tensor representation PyTorch expects.
  Why it matters: Thinking of images as numerical grids makes their tensor representation easier to understand.
• Pixel value precision: Describe how pixel values are often encoded as 8-bit integers, while medical, scientific, and industrial images may use higher precision such as 12 or 16 bits. Explain that higher precision can provide a wider range or greater sensitivity for measured physical properties.
  Why it matters: Pixel values can represent measurements, so their precision may matter as much as the image’s shape.

Key terms: pixels, grayscale image, scalar value, tensor representation, 8-bit integers, 12-bit, 16-bit, numerical precision

4.1.1 Adding color channels

This section explains how RGB represents image colors with three intensity channels and how to interpret each channel as a grayscale map.

• RGB color channels: Explain how RGB represents a color with red, green, and blue intensities, and describe what an individual color channel shows in an image.
  Why it matters: Reading the channels separately helps you understand how a color image is represented numerically.

Key terms: RGB, color channel, intensity, grayscale intensity map

4.1.2 Loading an image file

This section introduces loading an image with imageio and explains how the resulting array’s dimensions relate to the layout PyTorch image modules require.

• Image array dimension layout: Describe the dimensions of the loaded image array and distinguish their layout from the C × H × W layout required by PyTorch modules that handle images.
  Why it matters: An image can be loaded successfully yet still have its dimensions in the wrong order for a PyTorch image module.
• Choice of image loader: Explain why the example uses imageio and what kind of library output can be turned into a PyTorch tensor.
  Why it matters: The image-loading library is a practical choice, not a prerequisite for using the image data with PyTorch.

Key terms: img_arr, NumPy array-like object, red, green, and blue channels, C × H × W, imageio, TorchVision, NumPy array

4.1.3 Changing the layout

This section changes image tensors to a channel-first layout and shows how to store multiple images in a batch tensor.

• Channel-first image layout: Explain how permute changes an image from H × W × C to C × H × W, and why edits through either tensor affect the other.
  Why it matters: You can put image dimensions in the needed order cheaply, but changes to the shared data remain visible through both tensors.
• Batched image tensor layout: Describe how the section stores multiple RGB images in one tensor, including the batch dimension, shape, and element type used in its example.
  Why it matters: The batch layout keeps the image count, color channels, height, and width in distinct dimensions for neural-network input.

Key terms: permute, H × W × C, size, stride, underlying storage, batch, N × C × H × W, RGB, 8-bit integer

4.1.4 Normalizing the data

Explains why image tensors are converted to floating point and normalized, and compares two ways to scale pixel values.

• Image input scaling rationale: Explain why image tensors are typically cast to floating point and normalized before being used as neural-network inputs.
  Why it matters: This explains why preparing pixel values is part of preparing images for training.
• Pixel normalization alternatives: Distinguish dividing 8-bit pixel values by 255 from standardizing each channel, and describe which data should supply the standardization statistics in practice.
  Why it matters: The choice of scaling method determines how image values are presented to a network.

Key terms: floating-point tensors, normalize, 255, standardization, mean, standard deviation, channel

4.2 3D images: Volumetric data

This section explains how stacked CT slices become volumetric tensors and what their intensity values represent.

• CT volume tensor dimensions: Explain how stacking 2D CT slices represents 3D anatomy, and distinguish the depth dimension from an image’s color-channel dimension. State the tensor shape given for batched volumetric data.
  Why it matters: Keeping depth separate from channels makes the tensor layout reflect the anatomy it represents.
• CT intensity and channels: Describe what CT intensity represents and why native CT data may omit a channel dimension.
  Why it matters: It clarifies what the values in a CT volume mean and why raw data may lack an explicit channel axis.

Key terms: slice, volumetric data, depth, channel dimension, N × C × D × H × W, intensity, density, single intensity channel, native data formats

4.2.1 Loading a specialized format

Shows how to load a DICOM CT series as a volume and add the channel dimension expected by PyTorch.

• DICOM Series as a Volume: Describe how imageio’s volread loads a CT scan from a directory of DICOM files into a NumPy 3D array.
  Why it matters: This is the step that turns the scan’s files into a volume you can work with.
• CT Volume Channel Dimension: Explain why the loaded volume needs a channel dimension and how unsqueeze changes its shape; describe how multiple volumes could then form a batch.
  Why it matters: The shape makes the channel explicit and leaves room to batch several scans.

Key terms: volread, imageio, DICOM, NumPy 3D array, unsqueeze, channel dimension, batch direction, [D, H, W], [1, D, H, W]

4.3 Representing tabular data

This section describes the structure of tabular data and explains why its mixed column types must be represented numerically for PyTorch models.

• Tabular samples and columns: Describe a table as one row per sample or record, with each column holding one piece of information about that sample. Distinguish the assumed lack of meaningful row order from a time series, whose samples are related by time.
  Why it matters: This identifies what each part of a table represents before it is loaded as data.
• Mixed columns and numeric tensors: Explain why a table can contain different kinds of columns while a PyTorch tensor is homogeneous and typically numeric. State why numeric inputs suit neural networks, without specifying a conversion method.
  Why it matters: It explains why raw spreadsheet columns cannot always be treated as ready-made neural-network inputs.

Key terms: sample, record, column, independent samples, time series, tabular data, homogeneous, numerical values, labels, PyTorch tensors

4.3.1 Using a real-world dataset

Introduces the Wine Quality table as a real-world example of using chemical measurements to predict a sensory quality score.

• Wine quality features and target: Describe how the Wine Quality dataset separates chemical measurements from a sensory quality score, and explain the proposed task of predicting the score from those measurements.
  Why it matters: Identifying the inputs and the score to predict makes the purpose of the table clear before working with it as data.

Key terms: Wine Quality dataset, chemical variables, sensory quality score

4.3.2 Loading a wine data tensor

The section loads a wine CSV into a PyTorch tensor and explains what different kinds of numerical values do—and do not—mean.

• Wine CSV tensor loading: Describe how the wine CSV is loaded with NumPy and converted to a PyTorch tensor, including the specified data type, delimiter, and skipped header.
  Why it matters: This turns the file into data that can be inspected and worked with in PyTorch.
• Continuous value scales: Explain what makes a value continuous and distinguish ratio from interval scales using the section’s examples.
  Why it matters: It clarifies which comparisons between numerical measurements are meaningful.
• Ordinal and categorical values: Distinguish ordinal values from categorical values and explain why their assigned numbers cannot be treated as ordinary measurements.
  Why it matters: A number in a dataset does not necessarily license arithmetic—or even imply an order.

Key terms: CSV, NumPy array, 32-bit floating-point, torch.Tensor, quality score, continuous values, ratio scale, interval scale, ordinal values, categorical values

4.3.3 Representing scores

The section shows how a wine score can serve as either a continuous regression target or a classification label, kept separate from the model’s inputs.

• Score as target variable: Distinguish treating the wine score as a real-number target for regression from treating it as a label for classification, and explain why the score is kept separate from the chemical-analysis inputs.
  Why it matters: Keeping the answer out of the inputs lets the model learn to predict it from the chemical analysis.
• Integer encoding of labels: Describe how categorical target labels can be represented as an integer vector, including when the original labels are strings such as wine colors.
  Why it matters: Integer labels give categorical targets a tensor representation.

Key terms: score, continuous variable, regression task, label, classification task, ground truth, target tensor, labels, integer vector, string labels

4.3.4 One-hot encoding

This section explains when to represent wine scores as one-hot categories rather than integers and how to construct that representation with PyTorch.

• Integer and one-hot representations: Distinguish what integer scores express from what one-hot encoding expresses, and explain when each representation is appropriate for wine data.
  Why it matters: The representation should reflect whether values have meaningful order and spacing or merely identify categories.
• Scatter-based one-hot construction: Trace how scatter_ places a 1 in each row's category column and why the target indices are unsqueezed first.
  Why it matters: This connects the categorical representation to the tensor operation and shape needed to build it.
• Target and input score forms: Distinguish using class indices as training targets from using a score as a categorical network input.
  Why it matters: The required representation depends in part on whether the score is a target or a categorical input.

Key terms: one-hot encoding, ordering, equal intervals, categorical data, scatter_, index tensor, unsqueeze, singleton dimension, in place, class indices

4.3.5 When to categorize

The section explains the trade-offs in treating ordinal data as categorical or continuous, then shows how to normalize the wine data column by column.

• Ordinal data representation trade-offs: Explain why ordinal data has no general representation recipe and what is lost or introduced when it is treated as categorical or continuous.
  Why it matters: The representation you choose changes what the model is given about ordered values.
• Column-wise tensor normalization: Describe how the section normalizes the tensor of chemical-analysis variables using per-column means and standard deviations.
  Why it matters: Per-column normalization puts the chemical-analysis variables into a form that helps learning.

Key terms: ordinal, categorical, continuous, dim=0, reduction, mean, standard deviation, normalize

4.3.6 Finding thresholds

The section uses tensor masks to inspect wine-quality groups, tries a single-feature threshold, and checks how well its predictions match the actual ratings.

• Boolean masks for wine rows: Explain how a Boolean tensor selects rows of wine data and what happens when the mask for scores of 3 or less is applied.
  Why it matters: Boolean masks let you inspect the records belonging to a particular quality group.
• Grouped means and sulfur threshold: Describe how the section compares quality groups and turns an observed feature difference into a crude rule for predicting good wines.
  Why it matters: A group-level pattern can suggest a prediction rule, but that rule still needs testing against actual ratings.
• Threshold results and limitations: Explain how the section checks threshold predictions against actual good wines and why it considers the rule inadequate.
  Why it matters: Checking both correct predictions and missed good wines shows why an apparent feature difference is not enough for a reliable classifier.

Key terms: advanced indexing, torch.bool, bad_indexes, bad_data, target, good, middling, and bad categories, .mean(), total sulfur dioxide, threshold, midpoint

4.4 Working with time series

The section contrasts independent rows in a flat table with data whose order conveys information over time, introducing bike-sharing counts as the example.

• Row Order in Time Series: Distinguish a flat table whose rows are independent from data in which a time column lets you examine how values change from one period to the next.
  Why it matters: Time information changes what relationships you can look for in otherwise tabular data.

Key terms: flat table, year column

4.4.1 Adding a time dimension

The section explains why hourly bike-sharing data should be reshaped to preserve time order and separate days, hours, and recorded variables for neural-network input.

• Day Hour Feature Dimensions: Describe how the bike-sharing data should change from one row per hour to separate axes for day, hour of day, and recorded variables.
  Why it matters: The choice of axes determines how the model receives each fixed-size portion of the time series.
• Value of Timepoint Ordering: Explain why successive hourly rows can be more useful than treating each row independently, using the section’s bike-rental example.
  Why it matters: Preserving the order makes earlier observations available as context for later predictions.

Key terms: time-series data, day, hour of the day, columns of data, time-series dataset, successive timepoints, ordering, causal relationships across time

4.4.2 Shaping the data by time period

This section reshapes hourly bike-sharing data into daily tensor sequences and explains why the chosen dimension order and continuous time periods matter.

• Daily sequence dimension order: Explain what N, C, and L represent in the daily bike-sharing tensor and why the section chooses N × C × L rather than N × L × C.
  Why it matters: The dimension order determines how easily you can access a variable’s values across a day.
• Storage-preserving daily reshape: Trace how the hourly bikes tensor becomes daily sequences: view it with day, hour, and column axes, then transpose to put columns before hours. Explain why view does not copy the stored data.
  Why it matters: It shows how to reorganize hourly observations into daily sequences without copying the data during the view.
• Requirements for time period chunks: Explain why forming fixed daily or weekly sequences requires appropriately ordered, gap-free hourly data and a row count divisible by the chosen period length.
  Why it matters: Without consecutive, complete hours, a 24-row chunk would not reliably represent one day.

Key terms: N × C × L, N × L × C, channels, sequences, view, storage, stride, transpose, daily_bikes, 24 hours

4.4.3 Ready for training

The section prepares hourly bike-rental data for training by encoding weather categories, handling the daily time-series layout, and rescaling quantitative variables.

• Ordinal weather representation: Distinguish treating the four-level weather situation as categorical labels from treating its ordered levels as values that can be scaled to 0.0–1.0.
  Why it matters: The representation determines whether the data presents weather levels as separate labels or retains their ordering numerically.
• Weather indicators in time tensors: Trace how the section creates one-hot weather columns and appends them to both an hourly matrix and a daily time-series tensor.
  Why it matters: Adding weather features without disturbing the hourly or daily layout makes the time-series tensor usable by a network.
• Quantitative variable rescaling: Describe the two rescaling options given for quantitative variables and the result of standard-deviation scaling.
  Why it matters: Rescaling puts quantitative inputs into a form the section identifies as beneficial for training.

Key terms: weather situation, ordinal, categorical, continuous variable, one-hot-encoded vector, one-hot encoding, scatter, unsqueeze, cat, daily_bikes

4.5 Representing text

This introduction explains why text must be represented as a tensor of numbers for neural networks and places that task in the context of modern NLP.

• Text as numeric tensors: Explain that text must be turned into a tensor of numbers before a neural network can process it, making data reshaping the first step toward NLP with PyTorch.
  Why it matters: The same PyTorch tools can address text tasks once the data is cast in a form a network can process.
• Shift to data-driven NLP: Distinguish earlier NLP pipelines with hand-encoded grammar rules from modern end-to-end training on large text corpora, where rules can emerge from data.
  Why it matters: This contrast explains why preparing text as network input matters in current NLP.
• Past information in text models: Describe the introductory distinction between recurrent neural networks, which repeatedly combine new input with previous model output, and transformers, which incorporate past information more flexibly.
  Why it matters: The distinction gives context for why the choice of architecture matters after text has been represented numerically.

Key terms: tensor of numbers, reshaping the data, multistage pipelines, end to end, large corpora, recurrent neural networks (RNNs), transformers, past information

4.5.1 Converting text to numbers

This subsection introduces character-level and word-level text processing and identifies one-hot encoding as a way to represent either in tensors.

• Text granularity and encoding: Distinguish character-level from word-level text processing, and explain that both can use one-hot encoding to put text into tensor form.
  Why it matters: The choice of character or word changes what the network treats as a unit, but not the encoding approach introduced here.

Key terms: character level, word level, one-hot encoding, tensor form

4.5.2 One-hot-encoding characters

The section explains character encodings, how to choose a useful character set, and how to represent each character in a line of text as a one-hot tensor row.

• Character Codes and Encodings: Distinguish a character’s numeric code from its representation in bits, using ASCII and Unicode as the section’s examples.
  Why it matters: Character codes provide the positions used in the section’s one-hot representation.
• Useful Character Set Selection: Justify limiting the characters included in a one-hot representation to those useful for the text and task.
  Why it matters: Choosing the character set keeps the representation focused on characters relevant to the text.
• One-Hot Character Tensor Layout: Describe the one-hot vector for a character and how such vectors are arranged for a line of text.
  Why it matters: This is the section’s concrete way to turn a line of characters into a tensor.

Key terms: ASCII, Unicode, UTF-8, UTF-16, UTF-32, character set, lowercase, one-hot encoding, letter_t, vector

4.5.3 One-hot encoding whole words

The section shows how to one-hot encode whole words, explains the tradeoff between character- and word-level representations, and introduces byte pair encoding as an intermediate approach.

• Whole-word one-hot encoding: Describe how to turn a sentence into a word-level one-hot tensor using cleaned words and a word-to-index dictionary.
  Why it matters: It connects the words in a sentence to the rows and positions of a tensor.
• Character and word tradeoff: Explain why character-level encoding uses fewer classes while word-level encoding is more informative but can be impractically wide.
  Why it matters: The tradeoff explains why neither individual characters nor whole words are an effortless choice of text unit.
• Byte pair encoding compromise: Describe byte pair encoding as an intermediate way to split text, including how its dictionary grows and how rare words can become subunits.
  Why it matters: It illustrates a way to choose text units between individual characters and whole words.

Key terms: clean_words, word2index_dict, one-hot-encoded vector, vocabulary, character-level, word-level, dictionary, byte pair encoding, subunits

4.5.4 Text embeddings

This section explains why word embeddings offer a compact, meaningfully structured alternative to one-hot encoding and contrasts classic embeddings with context-sensitive ones.

• Compact word representations: Explain why one-hot encoding becomes unwieldy for large vocabularies and how an embedding represents each word with a fixed-size vector of floating-point numbers.
  Why it matters: This explains why text needs a representation more manageable than an ever-growing one-hot vector.
• Meaningful embedding space: Describe why useful embeddings place words used in similar contexts near one another, using the hand-designed fruit, flower, and dog example to illustrate the idea.
  Why it matters: The arrangement of words, not just the short vector length, makes embeddings useful for later learning.
• Learned embedding relationships: Explain, at the level described here, how embeddings can be generated from text and what kinds of relationships their vectors can capture.
  Why it matters: The section’s manual map is an illustration; text-based training can create useful relationships without placing every word by hand.
• Context-sensitive word vectors: Distinguish the fixed word-to-vector mapping of simpler classic embeddings from the context-sensitive mapping described for more contemporary models.
  Why it matters: It prevents treating every word embedding as necessarily the same vector in every sentence.

Key terms: one-hot encoding, vocabulary, embedding, embedding vector, embedding space, context, neighboring regions, axes, corpus, neural networks

4.5.5 Text embeddings as a blueprint

The section uses text embeddings as a blueprint for representing categorical data and for thinking about co-occurrence and sequence problems beyond text.

• Embeddings for categorical data: Explain why embeddings are useful for large vocabularies and other categorical data, and relate them to one-hot encoding.
  Why it matters: This connects a text technique to a general way of representing categories.
• Learning and fine-tuning embeddings: Distinguish starting embeddings from random numbers in nontext applications from fine-tuning prelearned text embeddings for a task.
  Why it matters: Embedding vectors need not be fixed in advance; they can be improved for the problem being solved.
• Text methods beyond text: Describe how word embeddings and text processing offer examples for co-occurrence and sequence problems outside NLP.
  Why it matters: Ideas developed for text can suggest ways to approach other data with context or sequence structure.

Key terms: embeddings, one-hot encoding, embedding vectors, categorical data, prelearned embeddings, fine-tuning, word embeddings, co-occurrences, context, recommender systems

4.6 Conclusion

The conclusion recaps preparing common kinds of data for neural networks and notes that the book does not cover every data format.

• Data preparation for networks: Explain that common kinds of data can be loaded and shaped into tensors for a neural network to consume.
  Why it matters: This connects tensor representation to the data a neural network will use.

Key terms: tensors, shape, neural network

4.7 Exercises

4.8 Summary