liveBook · Manning

Fusing

As explained in LLMs in Production

Fusing is the technique of mounting remote object storage on a machine so it appears and behaves like a local filesystem.

Fusing mounts an object-storage bucket on a machine so it behaves like an external drive. For LLM serving, this lets application code load a model directly into memory instead of downloading it first. section 6.1.2

The convenience comes with operational trade-offs:

  • It simplifies storage access but still pulls objects from multiple machines behind the filesystem interface. section 6.1.2
  • Mounting storage in the same region and zone can improve performance, yet fusing has proved slow in practice—usually faster than download-then-load. section 6.1.2
  • FUSE-based libraries support major cloud providers, Ceph, MinIO, laptops, and edge devices. section 6.1.2

Loading a model from a fused bucket

A serving node mounts a bucket as a filesystem and reads the model directly into GPU memory. The interface resembles local-disk access, while object retrieval remains distributed underneath. section 6.1.2

Figure 6.3 Different strategies for storing LLMs and their implications at boot time. Often, we have to balance system reliability, complexity, and application load time.
figure

Compare fusing with other LLM-storage strategies through the reliability, complexity, and boot-time loading trade-offs. section 6.1.2

How LLMs in Production covers it

How deep it goes
Defines bucket mounting, explains distributed retrieval and regional optimization, and stops short of implementation details.
Examples
Uses model loading from a fused bucket and deployments on cloud, on-premises, laptop, and edge environments.
What you can do after
Operate/configure FUSE-based mounts and evaluate their latency trade-offs for model loading.
How it connects
Connects object storage, LLM loading, GPU memory, FUSE, Ceph, and MinIO.
Cover of LLMs in Production
liveBookmanning.com
Read LLMs in Production on liveBook Get the book at manning.com

This explainer was generated by Manning's knowledge graph from the text of LLMs in Production. The section links point to the parts of the book it draws on.