A large language model (LLM) is an AI model trained on large text corpora to generate or transform natural-language content. A chatbot answering questions about a dripping faucet or making tomato sauce is one application. Like other AI models, it learns patterns from historical data and uses them to produce outputs for new inputs. section 1.3
An AI model transforms inputs into outputs using an architecture and learnable parameters such as weights and biases. Training adjusts those parameters; deployment adds an inference pipeline and serving infrastructure. This is a general model lifecycle, not a description of a specific LLM architecture. section 1.3
Figure 1.2 A high level conceptual overview of AI systems in an industry setting. The diagram illustrates the key components typically required to build and deploy an AI model. Starting from left to right, input features and training data are closely linked, as both are fed into the model. The model architecture, which is the core of the system, includes trainable weights and other configuration parameters. Hyperparameters, which are not trainable, are used to define the learning process. The loss function guides model training by measuring error, while the optimizer (e.g., gradient descent) updates the weights based on this feedback. Operational and deployment components include the inference pipeline, model output (such as prediction scores and confidence intervals), version control, and model serving infrastructure.
This overview connects training components with inference and serving infrastructure for AI models generally. section 1.3
Evaluate a language model as part of a product, not just in isolation. Offline evaluations can catch regressions and safety issues before launch, but online data is needed to assess real user impact. A model can perform well against historical labels yet disappoint when latency, user behavior, and downstream experience come into play. section 6.0
When evaluating models in user-facing products, check for:
- Misaligned metrics: a local improvement such as click-through rate can coincide with lower satisfaction or higher latency. section 6.0
- Experiment complications: behavior can shift over time, inference costs can fluctuate, and cold starts or training-data bias can make results harder to interpret. section 6.4 section 6.5
- Feedback loops: model outputs affect user behavior and logged data, which can influence later model iterations. Increased engagement may reflect exposure rather than relevance. section 8.2
A recommendation feedback loop
A recommender that shows one genre more often may collect more clicks on that genre. Those clicks can reinforce the initial preference, making engagement look stronger while narrowing the user experience. This illustrates a general model-evaluation risk, rather than an LLM-specific example. section 8.2
























































































