How deeply does LLMs in Production cover DeepSpeed?

Describes major optimizations and installation constraints, but does not explain implementation internals.

DeepSpeed

DeepSpeed is an optimization library for distributed deep learning that provides training and inference efficiency features.

DeepSpeed is a distributed deep-learning optimization library for making training and inference faster and more scalable, especially on multi-GPU systems. It targets long or multimodal inputs, quantization, cached weights and inputs, and workloads scaling to thousands of GPUs. section 5.1.2

Use DeepSpeed when the workload needs:

Multi-GPU scaling, memory management, gradient checkpointing, or caching. section 5.1.2
Training and inference acceleration with quantization, while recognizing that accuracy may depend on the configuration. section 5.1.2
A Linux-oriented setup; Windows has partial support and requires additional inference configuration. section 5.1.2

How LLMs in Production covers it

liveBook manning.com

Read LLMs in Production on liveBook Get the book at manning.com

Also explained in

liveBook manning.com

Domain-Specific Small Language ModelsAn open-source deep learning optimization library that supports distributed training and inference and is used here to speed up and memory-optimize Transformer models.

Compare how 2 books explain DeepSpeed →

Related concepts

librarykind of · A software library is a reusable collection of code or components that other programs can incorporate or invoke to provide functionality.
PyTorchuses · PyTorch is an open-source machine-learning framework for Python that provides tensor computation, automatic differentiation, and tools for building and training neural networks.
dataset splituses · A dataset split is the division of a dataset into distinct subsets assigned different purposes, such as training, validation, or testing.
transformer architectureuses · A transformer architecture is a neural-network architecture that processes sequences or other structured inputs using attention mechanisms and related transformations.
distributed trainingimplements · Distributed training is the coordinated training of a machine-learning model across multiple computing devices, processes, or machines by distributing computation or data among them.

This explainer was generated by Manning's knowledge graph from the text of LLMs in Production. The section links point to the parts of the book it draws on.