Negative sampling
Negative sampling is a technique that selects a subset of unobserved or candidate examples as negative examples, often to contrast with observed positive examples.
Knowledge Graphs and LLMs in Action
Negative sampling is a training technique that selects unobserved or candidate relationships as negative examples instead of evaluating every possible relationship.
- How it is used
- Reduces graph link-prediction cost while handling the imbalance between observed edges and overwhelmingly many nonedges.
- How deep it goes
- Explains sampled negatives, sigmoid scoring, cross-entropy terms, sampling distributions, and balancing weights without deriving their optimization.
- Examples
- Uses a biomedical knowledge-graph fact about aspirin treating headache and a sampled COX-2 alternative.
- What you can do after
- Evaluate link-prediction losses that score observed edges and sampled false relationships with tunable negative weighting.
Read how Knowledge Graphs and LLMs in Action explains negative sampling →
Graph Neural Networks in Action
A training technique that selects a subset of nonexistent relationships as negative examples instead of evaluating all possible negative examples.
- How it is used
- Balances positive and nonexistent edge labels during graph-autoencoder link-prediction training.
- How deep it goes
- Explains per-epoch sampling, edge concatenation, zero-label construction, loss calculation, and evaluation without sampling.
- Examples
- Uses a PyTorch Geometric training function that samples sparse negative edges and trains for repeated epochs.
- What you can do after
- Apply it via PyTorch Geometric to construct sampled edge-label batches for graph-autoencoder training.
Read how Graph Neural Networks in Action explains negative sampling →
Related concepts
- Word2Vecused by · Word2Vec is a family of shallow neural models that learns vector representations of words from their contexts in text.
- cross entropy lossuses · Cross-entropy loss measures the expected negative log probability that a predicted distribution assigns to outcomes drawn from a target distribution.
- class imbalanceaddresses · Class imbalance is a condition in a classification dataset where classes have substantially unequal numbers of examples.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.