liveBook · Manning

Deployment

As explained in Machines that Think

Deployment is the process of making a computational model available for operation in a target environment.

Deployment makes a trained model usable in a real environment. For large language models, this means resolving a practical tension: the scale that supports rich capabilities can make models costly and difficult to operate. Models may need to be reshaped, constrained, or compressed before they can run within real-world limits. section 4.3

Operating a model brings ongoing inference costs, latency requirements, memory demands, and the need to handle peak concurrent usage. Replicating a model across regions can improve reliability, compliance, or response time, but adds infrastructure costs. These pressures make deployment economics—not just model capability—a practical factor in how products use AI. section 4.6.2

Common deployment choices reflect these constraints:

  • Use a hosted model API or run an open model in a managed cloud environment. section 4.6.2
  • Choose infrastructure with attention to cost, response time, throughput, capacity, and the software’s fit with the hardware. section 4.6.2 section 8.1.3
  • Account for how compute access and platform partnerships can tie model development and operation to particular cloud providers. section 8.1.3

Hardware-specific optimization can improve speed and cost, but may make systems harder to move between platforms without losing efficiency. Deployment is therefore not just the final step of making a model available: its infrastructure choices can shape costs, portability, and future access to compute. section 8.1.3

How Machines that Think covers it

How deep it goes
Explains inference costs, serving demands, and infrastructure trade-offs without detailing implementation procedures.
Examples
Cites hosted model APIs, managed cloud models, regional replicas, and deployments using GPUs or TPUs.
What you can do after
Evaluate deployment choices by weighing operating cost, latency, throughput, capacity, and portability.
How it connects
Connects deployment to inference, cloud computing, APIs, hardware optimization, and compute access.

Also explained in

Compare how 29 books explain deployment →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Machines that Think. The section links point to the parts of the book it draws on.