What you will learn in Self-Hosting LLMs at Scale
Deploying and serving models with vLLM and Kubernetes
Chapter by chapter
Chapter 1: Why companies self-host large language models
This introduction contrasts provider-hosted LLM APIs with self-hosting and frames self-hosting as an increasingly accessible infrastructure task.
What you learn in chapter 1 →Concepts explained in this book
- self-hostingA deployment in which an organization runs and manages a model within infrastructure it controls.
- open-weight modelA model whose trained parameters are made available for others to download or use, without necessarily making its training data or code open.
- inference engineAn inference engine is software that loads a trained machine-learning model and computes outputs for input requests.
- mixture of expertsA mixture of experts model is a model whose expert components are selectively activated for each token by a routing component.
- inference platformA system surrounding a machine-learning model that serves it through an API and provides operational functions such as routing, protection, and monitoring.
- dense modelA dense model is a neural network in which the model’s parameters participate in computation for each input, rather than selecting a sparse subset.
- vLLMvLLM is an open-source inference and serving engine for large language models, designed to serve many users efficiently.
- large language modelA machine learning model trained on large text collections to understand and generate language.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.
