Reconstruction

As explained in Math and Architectures of Deep Learning

The process of producing an output intended to match the original input from a compressed representation.

Reconstruction produces an output intended to match an original input after information passes through a compressed representation. Because compression removes degrees of freedom, reconstruction is generally approximate and lossy: it preserves salient structure while discarding subtle variation. section 10.5.1 section 9.1.4

In an autoencoder, an encoder maps an input to a smaller embedding, while a decoder maps that embedding back to a reconstructed input. Training minimizes the mismatch between the original and reconstructed tensors, commonly with Euclidean distance or binary cross-entropy. section 10.5.1 section 9.1.4

The reconstruction workflow connects representation size, loss, and model architecture:

• Compress the input into an embedding with fewer degrees of freedom. section 10.5.1
• Expand the embedding into a tensor with the input's size, using transposed convolutions when appropriate. section 10.5.1
• Measure the difference between input and output, then optimize encoder and decoder weights to reduce it. section 10.5.1 section 9.1.4

For normalized image or vector values in [0, 1], binary cross-entropy compares each input element with its reconstruction. Summing over pixels gives a reconstruction objective whose minimum occurs when the output matches the input, although the loss need not equal zero there. section 9.1.4

PCA reconstruction

A three-dimensional dataset can be centered, projected onto two principal directions, padded back to three dimensions, and transformed through the principal-direction matrix. The resulting points approximately reconstruct the originals because the discarded dimension contains lost information. section 14.4.1

Figure 14.7 Geometric depiction of VAE latent-space modeling distributions

A geometric view shows a VAE decoder mapping a latent vector back into input space as a reconstructed point. section 14.7.1

How Math and Architectures of Deep Learning covers it

How deep it goes

Explains lossy compression, encoder-decoder structure, reconstruction losses, VAE geometry, and a PyTorch PCA implementation.

Examples

Uses image autoencoders, VAE latent-space geometry, and PyTorch PCA recovery of three-dimensional points.

What it assumes

Assumes familiarity with embeddings, neural-network encoders and decoders, tensor losses, and dimensionality reduction.

What you can do after

Apply reconstruction via autoencoder losses or PyTorch PCA to recover approximate inputs from lower-dimensional representations.

How it connects

Connects reconstruction to embeddings, encoders, decoders, transposed convolution, autoencoders, VAEs, binary cross-entropy, and PCA.

liveBookmanning.com

Read Math and Architectures of Deep Learning on liveBook Get the book at manning.com

Related concepts

• decodeproduced by · Decoding is the process of transforming an encoded or internal representation into an interpretable or target-form output.
• memoryimplements · Memory is a storage resource or mechanism that retains data or computational state for access during computation or later use.
• autoencoderused by · An autoencoder is a neural network that encodes input data into an internal representation and decodes it to reconstruct the input or an approximation.
• encode/decode processproduced by · The autoencoder workflow that maps input data to a lower-dimensional representation and then reconstructs it back into input space.

This explainer was generated by Manning's knowledge graph from the text of Math and Architectures of Deep Learning. The section links point to the parts of the book it draws on.