liveBook · Manning

What you will learn in Self-Hosting LLMs at Scale

Deploying and serving models with vLLM and Kubernetes

by Arek Borucki · Manning Early Access Program (MEAP)

Chapter by chapter

Chapter 1: Why companies self-host large language models

This introduction contrasts provider-hosted LLM APIs with self-hosting and frames self-hosting as an increasingly accessible infrastructure task.

1.1 Understanding the shift from API calls to owned infrastructure · 1.1.1 Force 1: Data privacy and confidentiality · 1.1.2 Force 2: Cost at scaleCost at scale · 1.1.3 Force 3: Control · 1.1.4 Force 4: Vendor concentration risk · 1.1.5 Force 5: Regulation · 1.1.6 Why self-hosting is possible now · 1.1.7 The major families · 1.1.8 The axis that matters most: dense versus Mixture of Experts · 1.1.9 Sizes and what they mean in practice · 1.2 What "self-hosting" actually means · 1.3 Surveying the open-weight model landscape · 1.4 Distinguishing open-weight from open-source, and why licenses matter · 1.5 Sketching the anatomy of an inference platform · 1.6 Deciding when self-hosting is the wrong choice · 1.7 What you’ll need for this book · 1.8 Summary

What you learn in chapter 1 →

Concepts explained in this book

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.