liveBook · Manning

CodeGen

As explained in Domain-Specific Small Language Models

A pretrained code-generation transformer model family used for Python code generation and inference optimization.

CodeGen is an open-source pretrained Transformer family for conversational program synthesis: users provide natural-language specifications, and the model generates subprograms. The mono family targets Python, making it a focused example of specialized code-generation models rather than a general-purpose assistant. section 7.2.1

Choose the model family and size according to language coverage, quality, latency, and deployment constraints:

  • Use multi for C, C++, Go, Java, JavaScript, and Python generation; use mono for Python-only generation. section 7.2.1
  • Select among 350M, 2B, 6B, and 16B parameter checkpoints; 350M is fast but unsuitable for production, while 2B is a stronger candidate. section 7.2.1
  • Treat inference optimization as a deployment path: convert the model to ONNX, then apply 8-bit quantization when CPU speed matters. section 7.4

Python inference and deployment comparison

A practical workflow loads Salesforce/codegen-350M-mono with Hugging Face Transformers, generates code from a prompt such as a function signature, and measures repeated inference runs. The same model can be exported through Optimum to ONNX and benchmarked against the PyTorch version; the material reports about 19% lower average inference time after conversion and a further 60% reduction after 8-bit quantization without sacrificing quality. section 7.2.1 section 7.4

import torch
from transformers import AutoTokenizer, CodeGenForCausalLM

device = "cpu"
model_id = "Salesforce/codegen-350M-mono"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = CodeGenForCausalLM.from_pretrained(model_id).to(device)
model.eval()

Inspect the minimal Transformers setup for loading the CodeGen tokenizer and causal language model on a selected device. section 7.2.1

How Domain-Specific Small Language Models covers it

How deep it goes
Explains architecture, model variants, generation, ONNX conversion, quantization, and evaluation, while deferring detailed output-quality analysis.
Examples
Demonstrates function completion, matplotlib charts, Dash applications, tensor creation, list mutation, and generated-code unit tests.
What you can do after
Apply it via Transformers, export it with Optimum to ONNX, quantize it, and benchmark inference across runtime configurations.
How it connects
Connects Transformer models with Hugging Face Transformers, ONNX, quantization, AST parsing, HumanEval, and unit testing.
Cover of Domain-Specific Small Language Models
liveBookmanning.com
Read Domain-Specific Small Language Models on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Domain-Specific Small Language Models. The section links point to the parts of the book it draws on.