A small language model (SLM) is a compact language model with millions to a few billion parameters. It trades broad knowledge and nuanced generation for speed, lower infrastructure demands, and effective performance on focused predictive or generative tasks. section 5.3.4 section 5.0
Choose an SLM when the task emphasizes controlled delivery rather than maximum general capability:
- Use it for lightweight, specific work such as topic classification, where faster training, simpler troubleshooting, and lower resource use support budget-conscious scaling. section 8.2.2
- Prefer it for high-volume tasks such as competitor-content analysis, where responsiveness and operating cost matter more than rich contextual understanding. section 5.3.4
- Avoid relying on it for deeply personalized brand messages or industry-specific reports requiring nuanced, contextually rich output. section 5.3.4
- Evaluate several open-source candidates, such as DistilBERT, Phi-2, and T5, before selecting a model for fine-tuning. section 8.2.2
Topic filtering for a content-generation app
A startup can use an SLM to classify internal documents against each client’s topic taxonomy, then restrict generation to documents carrying the requested topic label. A balanced labeled dataset supports supervised fine-tuning, while the smaller model keeps experimentation and multi-client scaling manageable. section 8.2.2 section 5.3.4
Table 8.2 Data sample for supervised fine-tuning
| Input |
Target output |
|
| Document title |
Text excerpt |
Label |
| “API integration guide” |
“Detailed steps to integrate workflows . . .” |
Workflow automation |
| “Compliance checklist 2024” |
“Ensures adherence to data protection . . .” |
Compliance and security |
| “Top-10 trends in IT governance” |
“In this article, we will present the top trends in compliance . . .” |
Compliance and security |
| “AI trends in automation” |
“Examining the latest advances in ML . . .” |
Cutting-edge automation |
Inspect labeled inputs, excerpts, and target topic categories used to prepare supervised fine-tuning data. section 8.2.2
