liveBook · Manning

Autoregressive model

As explained in Build AI Drug Discovery Pipelines

Autoregressive generation produces a sequence one element at a time, conditioning each prediction on previously generated elements.

An autoregressive model generates a sequence one token at a time. Each prediction conditions on the tokens already produced. For molecular generation, a unidirectional decoder builds a SMILES string from left to right until it emits an end-of-sequence token. section 10.4.3

A VAE-based molecular generation workflow uses autoregression as its decoding stage:

  • Tokenize a SMILES string with SmilesPE and encode it with a bidirectional GRU. section 10.4.5
  • Sample a latent vector z from the encoder's latent distribution using the reparameterization trick. section 10.4.5
  • Evaluate generated molecules with validity, uniqueness, novelty, and diversity metrics. section 10.4.5

Training can use teacher forcing: the decoder receives ground-truth tokens rather than its own previous predictions. This improves stability and convergence but creates exposure bias, because inference supplies no ground truth. Word dropout exposes the model to imperfect inputs and improves recovery from errors. section 10.4.3

At inference, greedy decoding selects the most probable token and is fast but can produce suboptimal sequences. Temperature sampling trades conservatism for diversity: lower values narrow choices, while higher values broaden them. section 10.4.3

How Build AI Drug Discovery Pipelines covers it

How deep it goes
Explains token-by-token decoding, decoder directionality, teacher forcing, exposure bias, dropout, and sampling strategies.
Examples
Uses SMILES reconstruction and novel-molecule generation with greedy decoding or temperature sampling.
What it assumes
Assumes tokenized SMILES and latent vectors while explaining sequence direction, teacher forcing, and inference behavior.
What you can do after
Build it from scratch with a unidirectional GRU decoder and generate molecules using greedy or temperature-based sampling.
How it connects
Connects SMILES, recurrent neural networks, GRUs, variational autoencoders, latent spaces, teacher forcing, and temperature sampling.

Also explained in

Compare how 8 books explain autoregressive model →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build AI Drug Discovery Pipelines. The section links point to the parts of the book it draws on.