liveBook · Manning

Tensor

As explained in Retrieval Augmented Generation, The Foundational Ideas

A tensor is a multidimensional numerical array used to represent data and intermediate computations in machine-learning models.

A tensor is a structured numerical representation used to carry model data and intermediate states. In Fusion-in-Decoder, the encoder returns a tensor shaped [N, sequence_length, hidden], containing one representation for each independently encoded question–passage sequence. section 5.2

Stacking and fusion

For N retrieved passages, the encoder produces N separate tensor representations. Fusion reshapes them into one sequence of length N × sequence_length; it does not sum or otherwise merge their values. The decoder then reads this long representation through cross-attention. section 5.2

Figure 5.2 The FiD architecture: Independent Encoding, Fused Decoding.

Use the architecture figure to trace how separate encoder tensors become one decoder-visible representation. section 5.2

How Retrieval Augmented Generation, The Foundational Ideas covers it

How deep it goes
Explains tensor shape and reshape-based fusion, but stops short of general tensor operations or numerical implementation details.
Examples
Uses N question–title–passage sequences and an [N, sequence_length, hidden] encoder output.
What you can do after
Evaluate/understand how FiD reshapes encoder tensors before decoder cross-attention.
How it connects
Connects tensors to encoders, decoders, self-attention, cross-attention, and sequence representations.

Also explained in

Compare how 10 books explain tensor →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Retrieval Augmented Generation, The Foundational Ideas. The section links point to the parts of the book it draws on.