CrystaLLM is a decoder-only Transformer language model that generates inorganic crystal structures from textual crystallographic information files (CIFs). It replaces conventional chemical representations with CIF symbol sequences, targeting physically plausible structures while reducing reliance on computationally intensive candidate optimization. section 8.4
During training, the model tokenizes CIFs and predicts each next token with a causal language-model objective. Cross-entropy compares predictions with left-shifted targets; tokens include CIF tags, atoms, digits, and punctuation. During inference, autoregressive sampling continues from a starting sequence until termination. Monte Carlo tree search and a graph-based neural network improve structural sampling. section 8.4
Figure 8.3 The CrystaLLM training process (a) and the generation process with MCTS decoding (b)
Use the training-and-inference diagram to connect causal next-token prediction with MCTS-based CIF generation. section 8.4
A practical generation workflow:
- Create a prompt containing a target composition, such as data_Na2Cl2. section 8.4
- Sample CIF text with a checkpoint, controlling top-k selection, output count, token length, and device. section 8.4
- Postprocess and evaluate generated files for space-group consistency, atom-site multiplicity, bond-length reasonableness, and validity. section 8.4 section 8.5
- Use MCTS decoding when improved sampling quality is preferred over straightforward autoregressive sampling. section 8.4
Generating Na2Cl2 structures
A composition prompt for Na2Cl2 can produce CIF output containing symmetry metadata, lattice dimensions, chemical formulas, and atom-site coordinates. Two generated samples achieved consistent space groups and atom-site multiplicities, reasonable bond lengths, and full validity under the reported evaluator. section 8.4
