liveBook · Manning

Open-weight model

As explained in Self-Hosting LLMs at Scale

A model whose trained parameters are made available for others to download or use, without necessarily making its training data or code open.

An open-weight model has trained parameters, or weights, available to download and run. That makes self-hosting possible, but it does not automatically make the model open-source or grant unrestricted use. Open-source AI also requires access to the components and information needed to study, modify, and reproduce the system. section 1.1.6 section 1.4

For a team considering self-hosting, the decision is not simply whether an open-weight model is good enough. It is which model fits the workload, what infrastructure it needs, and whether the organization can operate it reliably. Self-hosting can support privacy, cost, control, vendor-risk, or regulatory goals, but may not fit low-volume workloads or teams without platform ownership. section 1.1.6 section 1.8

A practical shortlist starts with these checks:

  • Review the license before testing quality. Custom licenses can restrict commercial use, training on outputs, or other activities; terms can differ within a family or change between generations. section 1.4 section 1.8
  • Check the upstream lineage of derivatives. A fine-tune incorporates base-model weights, while distillation can raise different questions about source-model terms and training data. section 1.4
  • Match the model and serving setup to the workload. Consider memory needs and load time, as well as what happens when a machine fails or responses slow down. section 1.1.6
  • Use popularity as a starting point, not proof of production fit. The landscape includes fine-tunes and quantized versions of original models, alongside many other derivatives. section 1.3
Table 1.2 License orientation
License category
Commercial use
Modification
Redistribution
Practical meaning
Example families
Permissively licensed weights, for example under Apache 2.0 or MIT
Permitted
Permitted
Permitted
Generally suitable for production, provided that the license conditions are followed.
Many Qwen and Mistral models, gpt-oss, Muse Glimmer
Open-weight with a custom license
Usually permitted
Depends on the license
Depends on the license
Depends on the license
Llama, Gemma, Kimi K3
Research or non-commercial
Usually not permitted
Limited
Limited
Not suitable for commercial production use.
Some research releases, such as models under the Mistral Research License

Compare permissive, custom, and non-commercial license categories before reviewing the exact terms for a candidate model and its upstream lineage. section 1.4

Choosing for a law firm

Meridian's IT team begins with an open-model search, then faces a landscape of original families and derivatives. Because license terms matter to the firm, a candidate's permissions and upstream lineage need review before it becomes a production choice. section 1.3 section 1.4

Serving tools also differ by deployment size. llama.cpp and tools built on it, including Ollama and LM Studio, are popular for a single machine; vLLM serves many users at once. Kubernetes can manage vLLM across multiple machines when one is no longer enough. section 1.1.6

How Self-Hosting LLMs at Scale covers it

How deep it goes
Defines open weights, distinguishes open-source AI, and develops selection and licensing checks without prescribing a universal configuration.
Examples
Uses Meridian's model search and license review alongside families such as Llama, Qwen, and Mistral.
What you can do after
Evaluate model candidates for workload fit, infrastructure needs, license restrictions, and upstream lineage before deployment.
How it connects
Connects open weights to self-hosting, inference engines, fine-tuning, quantization, distillation, and model licensing.

Also explained in

Compare how 3 books explain open-weight model →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Self-Hosting LLMs at Scale. The section links point to the parts of the book it draws on.