liveBook · Manning

Saxml

As explained in Deep Learning with JAX

An experimental serving system for JAX-, Paxml-, and PyTorch-based models used for inference.

Saxml, also called Sax, is an experimental serving system for inference on JAX, Paxml, and PyTorch models. It organizes deployment around a Sax cell or cluster with an admin server and model servers. section 12.1.3 section 2.8

The admin server tracks model servers, assigns published models to them, and helps clients find the server hosting a specific model. section 12.1.3

Use it to understand the shape of a serving setup:

  • Inference serving for JAX, Paxml, and PyTorch models. section 12.1.3
  • A cluster layout with one admin server plus multiple model servers. section 12.1.3
  • Model publication, assignment, and client lookup through the admin server. section 12.1.3

Deployment shape

A Sax cell is described as a server-side deployment unit: the admin server coordinates published models, while model servers actually serve them to clients. section 2.8

How Deep Learning with JAX covers it

How deep it goes
Explains the cluster roles and routing flow, but stops short of implementation or configuration details.
Examples
Uses the Sax cell or cluster as the concrete deployment example.
What it assumes
Assumes familiarity with model inference and basic deployment roles, while introducing Sax cell terminology from zero.
What you can do after
Operate a Sax cell by publishing models, serving them from model servers, and letting clients locate the right server.
How it connects
Connects to JAX, Paxml, PyTorch, inference, and model servers.
Cover of Deep Learning with JAX
liveBookmanning.com
Read Deep Learning with JAX on liveBook Get the book at manning.com

This explainer was generated by Manning's knowledge graph from the text of Deep Learning with JAX. The section links point to the parts of the book it draws on.