liveBook · Manning

Open-weight model

As explained in Machines that Think

An open-weight model is a machine-learning model whose learned parameter values are made available for others to download and run.

An open-weight model makes its trained parameters available for download and local execution. This gives users flexibility over deployment and often over modification, but also makes them responsible for more of the infrastructure and optimization. section 8.1.2

Open-weight does not necessarily mean open-source. Parameters may be available while training data, training code, license terms, or the full development process remain restricted. Availability of weights alone therefore does not establish what a user is permitted or able to change. section 8.1.2

Some smaller or heavily optimized models can run on a laptop. Production deployment at scale still requires servers, storage, scaling, and maintenance expertise. At low usage, those costs can exceed API access costs; self-hosting becomes economically competitive only at sufficiently large volumes to spread fixed costs. section 8.1.4

Open-weight models can be found through platforms such as Hugging Face. They can also be accessed through managed platforms and APIs, including services that use specialized hardware to accelerate inference. Downloading and running weights is therefore one deployment option, not the only way to use these models. section 8.1.2

Figure 8.1 The large language model ecosystem. Models are built on a layered system that begins with specialized hardware and extends through cloud infrastructure and model providers. Interaction with this system occurs primarily through the two bottom layers, where users either consume models via managed services that abstract the underlying complexity or deploy open-weight models on self-managed infrastructure, taking on greater control and responsibility.

Compare managed model services with self-managed open-weight deployment in the wider infrastructure ecosystem. section 8.1.2

How Machines that Think covers it

How deep it goes
Defines open-weight access and compares deployment costs and restrictions, without providing a self-hosting procedure.
Examples
Contrasts smaller or optimized models running on laptops with production deployments requiring servers, storage, and scaling.
What you can do after
Supports evaluating self-hosting against managed API access by weighing control, restrictions, infrastructure, and usage volume.
How it connects
Connects open-weight models to open-source, APIs, cloud infrastructure, and inference.

Also explained in

Compare how 3 books explain open-weight model →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Machines that Think. The section links point to the parts of the book it draws on.