liveBook · Manning

Tensor

As explained in Natural Language Processing in Action, Second Edition

A tensor is a multidimensional numerical array used to represent and transform structured data in machine-learning computations.

A tensor is a multidimensional numerical array that carries structured data through a machine-learning computation. Its shape describes the size of each dimension, while its data type describes the numbers it contains. In PyTorch, data from a pandas DataFrame can be converted to a torch.Tensor with a floating-point or integer dtype. section 7.2.6

Think of tensor shapes like the connections on building bricks: the output of one layer must fit the input expected by the next. For text, a tensor might organize a sentence by feature channel and word position. Adding another dimension lets the same representation include a batch of examples. section 7.2.6 section 7.5.1

Find a grammatical pattern

Represent each word using four part-of-speech channels: adverb, adjective, verb, and noun. A 4 × 3 kernel has one row per channel and one column per word; its weights mark the pattern adverb–verb–noun. Add a leading dimension to both the sentence tensor and kernel, then pass them to a PyTorch Conv1d layer. In the example, the output is 3.0 at “rightly timed pause,” where all three positions match. This kernel is handcrafted; a trained network can instead learn useful patterns through backpropagation. section 7.2.6

Listing 7.7 Converting a DataFrame to a tensor with the correct size
>>> import torch
>>> x = torch.tensor(
...     df.iloc[1:].astype(float).values,
...     dtype=torch.float32)             #1
>>> x = x.unsqueeze(0)        #2

Convert the numeric DataFrame rows to a float32 tensor and add the leading batch dimension with unsqueeze(0). section 7.2.6

Listing 7.8 Loading hardcoded weights into a Conv1d layer
>>> kernel = torch.tensor(kernel.values, dtype=torch.float32)
>>> kernel = kernel.unsqueeze(0)              #1
>>> conv = torch.nn.Conv1d(in_channels=4,
...                     out_channels=1,
...                     kernel_size=3,
...                     bias=False)
>>> conv.load_state_dict({'weight': kernel})
>>> print(conv.weight)

tensor([[[1., 0., 0.],
         [0., 0., 0.],
         [0., 1., 0.],
         [0., 0., 1.]]])

Load the expanded 4 × 3 pattern as the weights of a one-dimensional convolution layer. section 7.2.6

Track shapes as data moves through a network. In a CNN text classifier, token IDs become embedding vectors, convolutions shorten the sequence, and pooling prepares features for classification. For a padded sequence of length 40, a kernel of length 3 with stride 1 produces a convolution output of length 38 in the supplied configuration. section 7.5.1

Different architectures use tensor shapes differently. An RNN passes a hidden state from one token step to the next and can expose an output at each step. A transformer encoder instead passes an encoding tensor of consistent shape between layers; a sequence of embeddings is typically represented as a two-dimensional tensor. When inspecting model predictions, a PyTorch tensor can also be squeezed, detached, and converted to a NumPy array for an accuracy calculation. section 8.1.4 section 9.1.2 section 5.2.5

How Natural Language Processing in Action, Second Edition covers it

How deep it goes
Works through tensor conversion, dimensions, and CNN layer shapes; does not explain tensor storage internals.
Examples
Uses part-of-speech pattern matching and a CNN text classifier to demonstrate tensor construction and shape changes.
What you can do after
Apply it via PyTorch to convert tabular data, prepare convolution inputs, and inspect output shapes.
How it connects
Connects tensors to embeddings, convolution, recurrent neural networks, and transformer encodings.

Also explained in

Compare how 10 books explain tensor →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Natural Language Processing in Action, Second Edition. The section links point to the parts of the book it draws on.