Code generation uses an LLM to produce source code from prompts or other inputs. Code is a useful target because programming languages impose explicit rules, and generated output can be checked more objectively than ordinary prose. section 6.1 section 6.1.2
The model processes code as tokens. Case-sensitive names remain distinct, while comments and descriptive variable names connect programming instructions with human descriptions of intent. section 6.1
Generated code can be checked in several increasingly useful ways:
- Compile it into an executable; compilation catches a substantial portion of incorrect code. section 6.1.2
- Run unit tests that supply sample inputs and check whether the output is correct. section 6.1.2
- Use a syntax parser during generation to reject token sequences that would create syntax errors. section 6.1.2
Catching a Python syntax error early
After generating `if(A > B)`, a syntax checker can reject a newline as the next token because the statement requires a colon, prompting an alternative token instead. section 6.1.2
Figure 6.5 A Python code example where the current tokens if(A > B) have been generated. If the next token produced by the LLM is a newline, a syntax error will occur because an if statement must end in a colon to be valid. Running a syntax checker on each new token allows us to catch this error and force the LLM to pick an alternative token that doesn’t cause a syntax error.
Shows how token-by-token syntax checking can prevent an invalid continuation before generation finishes. section 6.1.2
