Inductive bias is the set of assumptions that helps a learning model prefer useful patterns instead of discovering every pattern from scratch. In molecular generation, the representation supplies part of this bias: character-level SMILES tokenization treats each character independently, while chemically meaningful tokens give the model a stronger starting point. section 10.4.4
Representation choices change what a model must learn:
- Character-level tokenization produces long sequences with weak chemical meaning, making the model learn structures one character at a time. section 10.4.4
- Smiles Pair Encoding (SPE), a chemistry-oriented Byte-Pair Encoding variant, merges frequent adjacent tokens into recurring motifs or functional groups. section 10.4.4 section 10.5
- Shorter, more meaningful sequences can produce more structured latent representations and reduce posterior collapse because the decoder has less opportunity to ignore the latent space. section 10.4.4
A benzene-ring representation
With character-level tokenization, the sequence c1ccccc1 is learned one character at a time, so the model must discover that the complete sequence represents a benzene ring. SPE can instead learn recurring chemical units as single tokens, giving the model a stronger structural starting point. section 10.4.4 section 10.5



