3 Large Language Model Operations: Building a platform for LLMs

 

This chapter covers

  • Overview of Large Language Models Operations
  • Deployment challenges
  • Large Language Models best practices
  • Required Large Language Model infrastructure

As we learned in the last chapter when it comes to transformers and Natural Language Processing (NLP), bigger is better, especially when it’s linguistically informed. However, bigger models come with bigger challenges because of their size, regardless of their linguistic efficacy, thus requiring us to scale up our operations and infrastructure to handle these problems. In this chapter we’ll be looking into exactly what those challenges are, what we can do to minimize them, and what architecture can be set up to help solve these challenges.

3.1 Introduction to Large Language Models Operations

3.2 Operations Challenges with Large Language Models

3.2.1 Long download times

3.2.2 Longer Deploy Times

3.2.3 Latency

3.2.4 Managing GPUs

3.2.5 Peculiarities of Text Data

3.2.6 Token Limits Create Bottlenecks

3.2.7 Hallucinations Cause Confusion

3.2.8 Bias and Ethical Considerations

3.2.9 Security Concerns

3.2.10 Controlling Costs

3.3 Large Language Model Operations Essentials

3.3.1 Compression

3.3.2 Distributed Computing

3.4 Large Language Models Operations Infrastructure

3.4.1 Data Infrastructure

3.4.2 Experiment Trackers

3.4.3 Model Registry

3.4.4 Feature Store

3.4.5 Vector Databases

3.4.6 Monitoring System

3.4.7 GPU Enabled Workstations

3.4.8 Deployment Service

3.5 Summary

sitemap