liveBook · Manning

Self-supervised learning

As explained in Timeless Algorithms, The Foundational Ideas

Self-supervised learning trains a model using supervisory signals derived from the data itself rather than manually supplied labels.

Manual labels can be costly, slow to produce, or inconsistent. Self-supervised learning derives a training task from the data itself: the model predicts information that is available but hidden from it, reducing reliance on human-supplied labels. section 11.7

A self-supervised task can ask a model to predict:

The hidden content serves as the target against which a prediction is evaluated. This approach does not remove the need for data; it reduces the constraint imposed by manual labeling and can make larger training collections usable. It is one development in neural-network representation learning, alongside broad-data training of models that can later be adapted to related tasks. section 11.7 section 11.8

How Timeless Algorithms, The Foundational Ideas covers it

How deep it goes
Defines data-derived targets and gives prediction examples without detailing model internals or optimization.
Examples
Uses missing-image prediction, next-element prediction, and prediction of concealed content as task examples.
What you can do after
Understand how data-derived prediction targets can reduce reliance on manually labeled training data.
How it connects
Links self-supervised learning to neural networks, learned representations, and foundation models.

Also explained in

Compare how 10 books explain self-supervised learning →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.