A mounted volume stores a large model on file-based storage attached at boot, letting a running service load it without packaging the model inside its image. section 6.1.2
Use the strategy when boot speed matters, but plan for its operational costs:
- Mounting a volume can provide very fast boot times and avoids downloading the model for every instance. section 6.1.2
- Deployment requires coordination so that each target region and zone has an appropriate volume. section 6.1.2
- Replication, backups, drive failures, and higher-cost SSDs create reliability and cost trade-offs. section 6.1.2
- A hybrid approach downloads the model during the first regional deployment, then lets later instances reuse the mounted volume. section 6.1.2
Model storage for new instances
A service downloads its model at boot and saves it to a volume mounted for later instances. The first deployment in a region gains no benefit, but subsequent instances can load the existing model directly. section 6.1.2
Figure 6.3 Different strategies for storing LLMs and their implications at boot time. Often, we have to balance system reliability, complexity, and application load time.
Compare mounted volumes with image-baked and object-store strategies by boot speed, reliability, complexity, and application load time. section 6.1.2
