Code generation turns instructions or other inputs into source code. Its apparent speed creates a dangerous shortcut: producing code is easy, but deciding whether it deserves to exist remains the engineering problem. section 8.1
Evaluate generated code rather than treating output as a finished result:
- Question whether the code will work when nobody is watching, especially in production conditions. section 8.1
- Treat pass rates, completion accuracy, and tokens per second as generation measures, not proof of correctness. section 8.1
- Keep evaluation central even when an assistant can produce functions or applications quickly. section 8.1
The attractive demo
A coding assistant generates a sorting algorithm, REST endpoint, or React component in a short demonstration. The output proves that code can be produced; it does not establish that the code should ship. section 8.1




















