Latent concatenation joins the latent vector to each token embedding before recurrent decoding. Every timestep therefore receives current token information and the complete molecular context, reducing the seq2seq information bottleneck. section 10.4.3
A decoder applies the technique by:
- Projecting the latent vector into the decoder's initial hidden-state shape for a warm start. section 10.4.3
- Embedding teacher-forced input tokens and expanding the latent vector across the sequence length. section 10.4.3
- Concatenating both tensors, processing them with a GRU, and projecting hidden states into token probabilities. section 10.4.3
Molecular SMILES reconstruction
In the VAE-CYC decoder, a latent vector representing a molecule accompanies every character embedding while the GRU reconstructs atoms, bonds, and special symbols. section 10.4.3
Figure 10.9 The 4-layer unidirectional GRU decoder used in the VAE-CYC implementation for generating SMILES molecular representations. The decoder is initialized with hidden states derived from the latent vector through linear projection. Concatenation of the latent vector with input character embeddings at every timestep resolves the information bottleneck problem inherent in standard seq2seq architectures. During training, ground truth characters are passed to the next layer instead of the decoder's own predictions (teacher forcing). The architecture processes characters sequentially with each GRU layer's output serving as input to the next layer at inference. The final layer outputs logits that are converted to probability distributions via softmax for next-character prediction, reconstructing the molecular structure X'.

Use the decoder architecture to trace latent initialization, per-timestep concatenation, teacher forcing, and next-character prediction. section 10.4.3
