Source dataset

As explained in Inside Deep Learning: Math, Algorithms, Models

The dataset domain on which a model is originally pretrained before being adapted to a new task.

A source dataset is the original, usually larger dataset used to pretrain a model before adapting it to a related target task. Its learned weights provide a starting point that can reduce training time and labeled-data requirements. section 13.0 section 13.2.1

Transfer works best when the source and target datasets share useful structure. For image models, pretrained convolutional filters can detect broadly useful edges, colors, and image changes, while a new output layer matches the target classes. section 13.0 section 13.2.1

ImageNet to cats versus dogs

ResNet-18 pretrained on ImageNet uses ImageNet as its source dataset. Replacing its final fully connected layer with two outputs adapts the model to cat-versus-dog classification; the convolutional layers retain pretrained weights. section 13.2 section 13.2.1

Figure 13.5 The tradeoff between source and target dataset size and relevance. The top-right corner is the best place to be, and top-left is sub-optimal and won’t always work. The bottom half is a uniformly difficult place to do effective transfer learning.

Compare source-dataset size and relevance to the target dataset when judging whether transfer learning is likely to work well. section 13.2.1

How Inside Deep Learning: Math, Algorithms, Models covers it

How deep it goes

Explains dataset similarity, pretrained features, and layer replacement, but stops short of detailing pretraining procedures.

Examples

Uses ImageNet-pretrained ResNet-18 adapted from 1,000 classes to cat-versus-dog classification.

What you can do after

Apply transfer learning via torchvision by loading pretrained ResNet-18 weights and replacing its final classification layer.

How it connects

Connects source datasets to transfer learning, target datasets, pretrained models, convolutional neural networks, and fine-tuned output layers.

liveBookmanning.com

Read Inside Deep Learning: Math, Algorithms, Models on liveBook Get the book at manning.com

Related concepts

• transfer learningused by · Transfer learning is a machine-learning technique that reuses knowledge learned from one task, domain, or dataset to improve learning or performance on another.

This explainer was generated by Manning's knowledge graph from the text of Inside Deep Learning: Math, Algorithms, Models. The section links point to the parts of the book it draws on.