Baking packages a model inside a Docker image, so every newly created container receives the model with the image and needs only one deployment artifact. section 6.1.2
Treat baking as a deliberate trade-off rather than a default storage strategy:
- Expect poor production boot-time gains: new machines still download the image, and downloading a model inside it may be slower than retrieving the model from object storage. section 6.1.2
- Accept significant security risk because containers can be easy to access, exposing the packaged model. section 6.1.2
- Manage two large assets instead of one: the model and the image. section 6.1.2
- Consider baking for edge deployments or other environments where one self-contained image substantially simplifies delivery. section 6.1.2
A self-contained edge service
An edge device can deploy one image containing both the service and its model, avoiding a separate model download while accepting larger images, weaker security, and potentially slower startup. section 6.1.2
Figure 6.3 Different strategies for storing LLMs and their implications at boot time. Often, we have to balance system reliability, complexity, and application load time.
Compare baking with other LLM storage strategies by balancing reliability, complexity, and application load time. section 6.1.2
