Contrastive learning
Contrastive learning is a representation-learning technique that trains a model to bring representations of related examples closer and separate representations of unrelated examples.
Retrieval Augmented Generation, The Foundational Ideas
Contrastive learning trains representations by bringing related examples closer and pushing unrelated examples farther apart in representation space.
- How it is used
- Supports scalable semantic similarity search and zero-shot dense retrieval by shaping a shared embedding space.
- How deep it goes
- Explains pairwise geometry, Siamese encoders, pooling, and indexing benefits, but does not specify a mathematical loss formula.
- Examples
- Uses Sentence-BERT similarity search and Contriever-powered HyDE retrieval as concrete applications.
- What you can do after
- Apply it via Siamese encoders and FAISS to precompute embeddings and perform fast similarity search.
Read how Retrieval Augmented Generation, The Foundational Ideas explains contrastive learning →
Build a Text-to-Image Generator (from Scratch)
Contrastive learning is a representation-learning technique that brings related examples closer and separates nonmatching examples in an embedding space.
- How it is used
- Uses shared embeddings to measure image–text similarity and support cross-modal retrieval.
- How deep it goes
- Explains encoders, batch similarities, and bidirectional losses, then stops short of deriving the loss equations.
- Examples
- Demonstrates Flickr 8k image–caption pairs, a beach-dog prompt, and batch diagonal versus off-diagonal similarities.
- What you can do after
- Apply it via CLIP to select images matching text descriptions and evaluate image–text compatibility.
Read how Build a Text-to-Image Generator (from Scratch) explains contrastive learning →
Outlier Detection in Python
A representation-learning technique that brings embeddings of similar items closer together and separates embeddings of dissimilar items.
- How it is used
- Uses learned embedding similarity as a self-supervised approach to tabular outlier detection.
- How deep it goes
- Defines the embedding objective and sketches ICL, but defers detailed implementation and training mechanics.
- Examples
- Uses altered images, tabular feature subsets, and KDD Cup records scored through DeepOD.
- What you can do after
- Apply it via DeepOD’s ICL estimator to fit tabular data and compute outlier scores.
Read how Outlier Detection in Python explains contrastive learning →
Also covered in
- Introduction to Generative AI, Second Edition
- Learn Generative AI with PyTorch
- Generative AI in Action
What to know first
- embeddingAn embedding is a numerical vector representation of an item in a feature space, encoding selected properties or relationships for computational tasks such as comparison or retrieval.
- computer visionComputer vision is the field of computing concerned with extracting and interpreting information from images, video, and other visual data.
- DALL-EDALL·E is a family of OpenAI generative models that creates images from natural-language descriptions and other conditioning inputs.
- diffusion modelA diffusion model is a generative model that learns to reverse a gradual noising process, generating samples through iterative denoising.
Related concepts
- CLIPused by · CLIP is a multimodal model and pretraining method that aligns image and text representations in a shared embedding space.
- Contrieverused by · Contriever is a pretrained dense-retrieval model that learns to represent queries and documents for unsupervised semantic retrieval.
- multimodalused by · A multimodal model is a model that processes, relates, or generates information in more than one data modality.
- self-supervised learninguses · Self-supervised learning is a machine-learning approach that derives training targets or supervisory signals from the data itself rather than relying on externally supplied labels.
Questions about contrastive learning
What is contrastive learning?
Contrastive learning is a representation-learning technique that trains a model to bring representations of related examples closer and separate representations of unrelated examples.
Which Manning books explain contrastive learning?
Retrieval Augmented Generation, The Foundational Ideas, Build a Text-to-Image Generator (from Scratch) and Outlier Detection in Python.
What should I know before learning contrastive learning?
Manning books that teach contrastive learning usually introduce embedding, computer vision, DALL-E and diffusion model first.
What kinds of contrastive learning are there?
Kinds of contrastive learning include ICL.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.


