How deeply does Knowledge Graphs and LLMs in Action cover negative sampling?

Explains sampled negatives, sigmoid scoring, cross-entropy terms, sampling distributions, and balancing weights without deriving their optimization.

Negative sampling

As explained in Knowledge Graphs and LLMs in Action

Negative sampling is a training technique that selects unobserved or candidate relationships as negative examples instead of evaluating every possible relationship.

Negative sampling makes graph link-prediction training practical by evaluating sampled false relationships instead of every possible node pair. It addresses both the computational cost of dense pairwise comparison and the severe class imbalance created by sparse graphs. section 11.4.1

A typical negative-sampling loss pipeline:

Score observed edges with a decoder and convert scores into probabilities with a sigmoid function. section 11.4.1
Sample candidate false relationships from a negative-example distribution rather than enumerating all nonedges. section 11.4.1
Combine positive and negative cross-entropy terms, using a balancing parameter to control the influence of negatives. section 11.4.1

Biomedical knowledge graph

For the fact “aspirin treats headache,” training rewards a high decoder score for the observed relationship while penalizing sampled alternatives such as “COX-2 treats headache.” section 11.4.1

How Knowledge Graphs and LLMs in Action covers it

Cover of Knowledge Graphs and LLMs in Action

liveBook manning.com

Read Knowledge Graphs and LLMs in Action on liveBook Get the book at manning.com

Also explained in

liveBook manning.com

Graph Neural Networks in ActionA training technique that selects a subset of nonexistent relationships as negative examples instead of evaluating all possible negative examples.

Compare how 2 books explain negative sampling →

Related concepts

cross entropy lossuses · Cross-entropy loss measures the expected negative log probability that a predicted distribution assigns to outcomes drawn from a target distribution.
Word2Vecused by · Word2Vec is a family of shallow neural models that learns vector representations of words from their contexts in text.
class imbalanceaddresses · Class imbalance is a condition in a classification dataset where classes have substantially unequal numbers of examples.

This explainer was generated by Manning's knowledge graph from the text of Knowledge Graphs and LLMs in Action. The section links point to the parts of the book it draws on.