Self-supervised learning

As explained in Generative AI in Action

A machine-learning technique that creates supervisory signals from the data itself to train a model without manually labeled examples.

Self-supervised learning creates supervisory signals from data itself, allowing models to learn without manually labeled examples. It supports foundational models that learn general representations from large text, code, image, audio, and 3D datasets. section 2.1

The technique contributes to adaptable AI systems by:

• Learning representations that transfer across tasks rather than serving only one narrowly defined task. section 2.1
• Supporting applications such as machine translation, question answering, object recognition, scene understanding, and robotic navigation. section 2.1
• Working with unstructured and spatial data, including text, code, images, audio, point clouds, medical scans, and 3D models. section 2.1

Cross-task model reuse

A foundational model can learn broad language representations from extensive text and code, then support translation or question answering even when those tasks were not its explicit training objective. section 2.1

Figure 2.1 Foundational model overview

Use the foundational-model overview to relate self-supervised learning to deep learning, transfer learning, and generative AI. section 2.1

How Generative AI in Action covers it

How deep it goes

Defines the technique and its role in representation learning but does not detail training objectives or implementation steps.

Examples

Uses translation, question answering, object recognition, scene understanding, and robotic navigation as application examples.

What you can do after

Evaluate/understand how models learn reusable representations from unlabeled multimodal data.

How it connects

Connects self-supervised learning with foundational models, transfer learning, deep learning, and generative AI.

liveBookmanning.com

Read Generative AI in Action on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  LLMs in ProductionSelf-supervised learning is a machine-learning approach that derives training targets from the input data itself rather than requiring manually supplied labels.
• 

  liveBookmanning.com
  Outlier Detection in PythonSelf-supervised learning is a machine-learning technique that derives training targets from the data itself rather than requiring manually assigned labels.
• 

  liveBookmanning.com
  The AI Pocket BookA training approach where a model learns from automatically generated targets derived from the input data itself.
• 

  liveBookmanning.com
  Inside Deep Learning: Math, Algorithms, ModelsA training setup that derives supervisory signals from the data itself rather than from human-provided labels.
• 

  liveBookmanning.com
  Build a Large Language Model (From Scratch)A training approach in which the model derives supervision signals from the input data itself rather than from human-provided labels.
• 

  liveBookmanning.com
  Deep Learning with R, Third EditionSelf-supervised learning is a machine-learning technique in which training targets are derived from the input data rather than supplied as manual labels.
• 

  liveBookmanning.com
  AI-Powered SearchSelf-supervised learning is model training in which supervisory targets are derived from the input data itself rather than supplied as external labels.

Compare how 8 books explain self-supervised learning →

Related concepts

• language modelused by · A language model is a computational model that represents patterns in linguistic tokens to estimate sequence probabilities and predict or generate language from context.
• machine learningkind of · Machine learning is a field of computational methods in which systems learn patterns or relationships from data to make predictions, decisions, or other outputs.
• pretrainingused by · Pretraining is the initial training of a model on broad data to learn general patterns or capabilities before adaptation to specific tasks or domains.
• unlabeled datauses · Unlabeled data consists of examples that lack target labels or task-specific annotations.
• foundational modelused by · A base or foundation model is a pretrained model that provides learned parameters or capabilities as a starting point for reuse or adaptation to downstream tasks.
• trainingused by · Training is a process of developing or improving a person’s or model’s capabilities through instruction, experience, or adjustment using data.

This explainer was generated by Manning's knowledge graph from the text of Generative AI in Action. The section links point to the parts of the book it draws on.