liveBook · Manning

Exposure bias

As explained in Build AI Drug Discovery Pipelines

Exposure bias is the mismatch caused when a sequence model is trained on ground-truth histories but must generate from its own possibly erroneous histories at inference.

Exposure bias is the mismatch between training and generation in sequence models: training uses ground-truth histories, while inference must continue from the model’s own predictions, including errors. In molecular generation, this affects decoders that reconstruct SMILES token by token. section 10.4.3

A teacher-forced molecular decoder follows this pattern:

  • A mistaken token can therefore place later predictions in histories the decoder rarely encountered during training. section 10.4.3

How Build AI Drug Discovery Pipelines covers it

How deep it goes
Defines the mismatch and shows its teacher-forcing context, but does not develop corrective methods.
Examples
Uses ground-truth SMILES tokens during training and sampled decoder tokens during molecule generation.
What it assumes
Assumes sequence tokens and decoder steps, while defining teacher forcing through the molecular-generation workflow.
What you can do after
Evaluate why a decoder can perform differently when generating from its own previous predictions.
How it connects
Connects sequence-to-sequence models, teacher forcing, autoregressive generation, GRU decoders, and SMILES representations.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build AI Drug Discovery Pipelines. The section links point to the parts of the book it draws on.