6 Generating Python Code

 

This chapter covers

  • What is the role of specialized Open Source LLMs for code generation.
  • How to optimize and quantize such LLMs to improve inference performance.
  • How to run such LLMs on a laptop after 4-bit quantization.

This chapter opens the section of this book dedicated to applying the concept learned so far to real-world examples of domain specific LLMs and their optimization and quantization. This one focuses on LLMs specialized in Python code generation and other programming assistance tasks. While majority of the available Open Source models falling into this category support multiple programming languages, here we are focusing on Python as target language, because it is the one which readers of this book are for sure familiar with and so everyone is in a good position when validating the quality of the results generated by the models under evaluation.

6.1 Transformers for programming language generation

6.2 Hands-on with Python language generation using a Transformer architecture

6.2.1 Python code generation with CodeGen

6.2.2 ONNX conversion and quantization of models not supported by Optimum

6.2.3 Model evaluation

6.2.4 Python code generation with better models

6.3 Inference (coding assistance) on commodity hardware

6.4 Summary