liveBook · Manning

Data residency

As explained in LLM Customization and Fine-Tuning

A requirement that data be stored or processed within specified geographic or organizational boundaries.

Data residency determines where a model runs and where sensitive data is stored or processed. Sending regulated data to a third-party API may violate compliance requirements; adaptation alone does not solve that problem. section 1.3 section 2.4

Treat residency as a deployment decision, separate from the choice of adaptation technique. A team can self-host whichever rung it selects, from LoRA to full supervised fine-tuning. section 1.7

Two deployment paths can keep data within a required boundary:

  • Use a provider’s tenant in the customer’s cloud region, with data-residency and no-training commitments. Azure OpenAI, Amazon Bedrock, and Google Vertex AI are named options. section 1.3
  • Self-host on organizational infrastructure, accepting responsibility for GPU capacity, serving, latency, and throughput. section 2.4

Hospital deployment

When patient data cannot leave the hospital, a team can start with a self-hosted open model, train a LoRA adapter on a few hundred curated clinical examples, and consider full fine-tuning if the adapter plateaus. section 1.7

Table 2.3 Buy versus build: criteria for choosing between a managed API and a self-hosted model.
Criterion
Buy: managed API
Build: self-hosted
Setup time
Minutes
Days to weeks to provision GPUs and configure the serving stack
Upfront cost
Low, pay per token
High, physical GPU hardware or cloud GPU VMs
Cost at scale
Grows linearly with usage
One-time infrastructure cost amortized across requests
Customization
Prompt engineering plus limited fine-tuning APIs
Fine-tuning, weight access, custom serving stack
Data privacy
Sent to vendor servers
Stays on your infrastructure
Model choice
Vendor decides versions and deprecations
Any open-weight model you choose
Latency control
Vendor SLA on shared infrastructure
You own the hardware and networking path
Maintenance burden
Zero infrastructure burden; vendor management overhead (deprecations, pricing, terms) is real.
Ongoing operational commitment for hardware, serving stack, and scaling

Compare managed APIs and self-hosting across privacy, setup time, cost, customization, and operational burden. section 2.4

How LLM Customization and Fine-Tuning covers it

How deep it goes
Distinguishes deployment-location controls from adaptation and names regional hosting and self-hosting options.
Examples
Uses hospital patient data and regional hosting from Azure OpenAI, Amazon Bedrock, and Google Vertex AI as examples.
What you can do after
Operate/configure deployment by choosing a managed regional tenant or self-hosting within the required boundary.
How it connects
Connects data residency to data privacy, self-hosting, managed APIs, and model adaptation.

Also explained in

Compare how 3 books explain data residency →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Customization and Fine-Tuning. The section links point to the parts of the book it draws on.