How deeply does Deep Learning with R, Third Edition cover self-supervised learning?

Defines input-derived targets and reconstruction objectives, but stops short of detailing optimization or implementation procedures.

Self-supervised learning

As explained in Deep Learning with R, Third Edition

Self-supervised learning is a machine-learning technique in which training targets are derived from the input data rather than supplied as manual labels.

Self-supervised learning trains a model with targets derived from the input data instead of manual labels. This makes vast amounts of unlabeled text, images, and other data available for training. section 1.8 section 1.14

Typical self-supervised objectives reconstruct or predict part of the original input:

Predict the next word in a sentence from its preceding context. section 1.8
Re-create a sharp image from a noisy version of that image. section 1.8
Reconstruct data from a partial representation, producing a broad representation that can later support downstream tasks. section 16.4.2

From reconstruction to general-purpose models

A foundation model can learn from broad internet-scale data by reconstructing missing or altered content. Afterward, the learned representation can be fine-tuned for many downstream tasks rather than training a separate model from scratch for each individual dataset. section 16.4.2

How Deep Learning with R, Third Edition covers it

Cover of Deep Learning with R, Third Edition

liveBook manning.com

Read Deep Learning with R, Third Edition on liveBook Get the book at manning.com

Also explained in

liveBook manning.com

LLMs in ProductionSelf-supervised learning is a machine-learning approach that derives training targets from the input data itself rather than requiring manually supplied labels.
liveBook manning.com

Outlier Detection in PythonSelf-supervised learning is a machine-learning technique that derives training targets from the data itself rather than requiring manually assigned labels.
liveBook manning.com

The AI Pocket BookA training approach where a model learns from automatically generated targets derived from the input data itself.
liveBook manning.com

Inside Deep Learning: Math, Algorithms, ModelsA training setup that derives supervisory signals from the data itself rather than from human-provided labels.
liveBook manning.com

Build a Large Language Model (From Scratch)A training approach in which the model derives supervision signals from the input data itself rather than from human-provided labels.
liveBook manning.com

AI-Powered SearchSelf-supervised learning is model training in which supervisory targets are derived from the input data itself rather than supplied as external labels.
liveBook manning.com

Generative AI in ActionA machine-learning technique that creates supervisory signals from the data itself to train a model without manually labeled examples.

Compare how 8 books explain self-supervised learning →

Related concepts

language modelused by · A language model is a computational model that represents patterns in linguistic tokens to estimate sequence probabilities and predict or generate language from context.
machine learningkind of · Machine learning is a field of computational methods in which systems learn patterns or relationships from data to make predictions, decisions, or other outputs.
pretrainingused by · Pretraining is the initial training of a model on broad data to learn general patterns or capabilities before adaptation to specific tasks or domains.
unlabeled datauses · Unlabeled data consists of examples that lack target labels or task-specific annotations.
foundational modelused by · A base or foundation model is a pretrained model that provides learned parameters or capabilities as a starting point for reuse or adaptation to downstream tasks.
trainingused by · Training is a process of developing or improving a person’s or model’s capabilities through instruction, experience, or adjustment using data.

This explainer was generated by Manning's knowledge graph from the text of Deep Learning with R, Third Edition. The section links point to the parts of the book it draws on.