Suppose a language feature needs private business data, predictable latency, or access where the internet is unreliable. An open-source model offers a starting point you can customize and deploy yourself instead of depending entirely on a vendor API. You usually do not need to train an LLM from scratch. section 1.2.2 section 1.3
Choose the model and customization method around the job:
- Start with a model suited to the task, then finetune on relevant data when its behavior needs to change. Evaluate more than task performance: bias and potential harm also matter. section 4.6
- Consider embeddings when the problem does not require generating answers with a full LLM workflow. section 4.6
- Use retrieval-augmented generation (RAG) when supplying up-to-date context is preferable to finetuning. section 10.0
A coding copilot
A Python-focused VS Code copilot can start with an open-source model already trained on coding problems. A RAG system supplies project context, avoiding a finetuning step and making that context easier to update. section 10.0
Self-deployment brings control, not a free pass. Compare it with a vendor service that may be faster to prototype and offer optimized production models and support. For either route, check whether the task warrants an LLM at all—especially if latency is tight or mistakes carry high risk. section 1.2.2 section 1.4
