Structural prior

Q: How deeply does Inside Deep Learning: Math, Algorithms, Models cover structural prior?

Defines structural priors and demonstrates CNN encoding through image tensors and model comparisons, but defers broader encoding methods.

As explained in Inside Deep Learning: Math, Algorithms, Models

An assumption about the organization of the data or task that is built into the model architecture before seeing data.

A structural prior is an assumption about how data or a task is organized, built into a model architecture before training. Neural networks become especially useful when their design encodes structure such as spatial correlation in images or sequential relationships in time-varying data. section 3.1

Choose an architecture that matches the structure you believe the data has:

Use fully connected layers for columnar data when feature order carries no special meaning. section 3.1
Use convolutional neural networks for images because nearby pixels are expected to be related. section 3.1

FashionMNIST baseline comparison

Represent 28×28 images as tensors and compare a flattened fully connected model with a CNN built from Conv2d and MaxPool2d layers. Train both with cross-entropy loss and compare accuracy; the CNN performs much better because its layers preserve and exploit image structure. section 6.0

How Inside Deep Learning: Math, Algorithms, Models covers it

Cover of Inside Deep Learning: Math, Algorithms, Models

liveBook manning.com

Read Inside Deep Learning: Math, Algorithms, Models on liveBook Get the book at manning.com

Related concepts

convolutional layeruses · A convolutional layer is a neural-network layer that applies shared learned filters to local regions of structured input to produce feature representations.
convolutional neural networkuses · A convolutional neural network is a neural-network architecture that applies learned convolutional operations to extract patterns from structured data.
attentionaddressed by · Attention is a neural-network mechanism that computes input-dependent weights to combine information according to its relevance to a computation.
image segmentationaddressed by · Image segmentation is a computer-vision task that partitions an image and assigns labels to its pixels or regions to delineate objects or meaningful areas.
prior probabilityincludes · A prior probability distribution represents beliefs or assumptions about an unknown quantity before relevant observations or evidence are incorporated.

This explainer was generated by Manning's knowledge graph from the text of Inside Deep Learning: Math, Algorithms, Models. The section links point to the parts of the book it draws on.