welcome
Thank you for purchasing the MEAP for Domain-Specific Small Language Models. To get the most benefit from this book, you’ll want to have some established skills in Python programming, with experience in PyTorch and knowledge about training/fine tuning Transformers-based language models, and willing to learn more about higher level and more user friendly API, optimizing such models, to make them ready for serving and inference on different hardware, also when on a tight infrastructure budget, or when the destination deployment system is a device, such as a laptop or a smartphone.
I started experimenting with Transformer models in early 2022, after reading DeepMind's original paper “Attention is All You Need” and did a first local implementation of a coding assistant based on a fine tuned version of the CodeGen Open Source model from SalesForce. That started first as a pet project, to learn more about this language model architecture, but then I spotted the potential for small language models in other more challenging and industry specific tasks and data (and still being of use in many projects in my specific field of application, biotech/pharma, since then). The discovery and understanding of optimization and quantization techniques and libraries contributed to the spark to move in this direction.