Chapter 10: Deploying and monitoring
This introduction explains why an LLM that succeeds in testing can still fail in production and why evaluation must be backed by operational infrastructure.
After this chapter you will be able to
- Production variability of LLMs. Explain why an LLM that works well in development may fail under real-world conditions, and how those failures differ from consistent software bugs.
- Evaluation and operational infrastructure. Explain why evaluation by itself cannot keep an LLM system running reliably and what kinds of infrastructure must act on its measurements.
- LLMOps production focus. Explain what LLMOps manages and how its focus differs from traditional MLOps.
- Reliability beyond uptime. Justify why a production LLM system needs continuous evaluation of outputs alongside operational monitoring.
- Production response processing layers. Describe how the architecture takes a user query through input processing, model execution, and output processing.
- Monitoring improvement feedback loop. Trace how monitoring and observability feed continuous improvement in the LLMOps architecture.
- Production model hosting choice. Explain that choosing between a hosted API and running an open source model on your own infrastructure is a foundational production decision, not merely a coding choice. Describe how costs, provider outages, and compliance requirements can affect that decision.
- Hosted API Starting Advantages. Explain why hosted APIs are attractive when building an LLM application, especially early in a project or with limited engineering resources.
- Usage Based Cost Growth. Explain why hosted-API costs estimated from short test prompts may rise sharply with real users and increasing traffic.
- Third Party Data Trade-Off. Describe the data-handling concern with hosted APIs and the compliance trade-off involved in running models locally instead.
- Hosted Model Control Limits. Explain how limited insight into hosted models and provider updates can affect production behavior.
- Self-hosting control tradeoff. Explain what teams gain by running an open source model in their own environment and why that control brings more operational responsibility.
- Production serving requirements. Describe the main infrastructure and operational capabilities this section says a production self-hosted LLM needs.
- Hybrid model routing. Explain how a hybrid service uses a small local model for common or short factual queries, sends nuanced or risky queries to a hosted LLM, and falls back to a hosted API when the local model fails or is overloaded.
- Production routing adaptability. Explain why keyword matching is brittle for production routing and describe the two alternatives given: semantic routing with embeddings and topic clusters, or a cheap LLM that classifies intent.
- Service health versus answer quality. Explain how a chatbot can appear healthy on a conventional dashboard while giving customers harmful, incorrect advice.
- Four production monitoring questions. Explain the four questions a team should ask daily about an LLM service: can customers get help quickly, are answers useful, do customers leave satisfied, and are costs under control. Describe why technical performance alone cannot answer the other three questions.
- Request logs for diagnosis. Explain why LLM request logs need to show more than HTTP status codes and total response times, and how time and cost information helps a team investigate anomalies.
- Token costs and priorities. Explain why the cost breakdown directs the team toward shorter outputs, more efficient prompts, and model routing instead of infrastructure changes when reducing spending.
- Contextual LLM alert signals. Explain why LLM alerts should focus on patterns and context rather than simple thresholds, using the section’s latency, error, and cost examples.
- Cost anomalies and actionable alerts. Explain how prompt changes, model-routing bugs, and shifts in user questions can raise LLM costs, and why alerts that include likely causes help teams investigate faster.
- Actionable dashboard questions. Explain the three questions the dashboard answers: what is happening now, where to investigate, and what the business impact is.
- Business outcome metrics. Distinguish metrics tied to customer success and unit economics from isolated technical or aggregate metrics, using the section’s examples.
- Layered Output Quality Checks. Explain how rule-based filters and an LLM-as-a-judge can help monitor outputs, while recognizing that quality is often subjective and context-dependent.
- Statistical Quality Drift Signals. Describe the response patterns the section recommends tracking for signs of quality drift and what a change in those patterns might suggest.
- User-centered success monitoring. Explain why good uptime, response times, and costs do not establish that an LLM is successful, and why monitoring should include explicit feedback and implicit behavioral signals.
- Voluntary feedback options. Explain why the team offered thumbs buttons, star ratings, and optional comments instead of requiring users to rate responses.
- Feedback participation trends. Explain why the team monitored feedback rates as well as the feedback users submitted.
- Behavioral signals of satisfaction. Describe how session abandonment, immediate rephrasing, copying a response, and return use can suggest frustration, usefulness, or trust. Explain why several immediate follow-up questions may signal confusion rather than satisfaction.
- Pattern-Based Feedback Triage. Explain how the team categorized and prioritized negative feedback, then looked for recurring patterns that suggested system changes rather than fixing complaints one at a time.
- Feedback and Business Outcomes. Describe how the team checked whether AI interactions delivered value beyond satisfaction ratings, using task completion, support-ticket volume, and customer retention.
- Billing Feedback Improvement Loop. Trace how billing escalations and user complaints led the team to identify a gap in complex-case handling, change routing, and measure the result.
- Convincingly Wrong LLM Answers. Explain how an LLM answer can look authoritative while containing errors, and why this kind of failure can damage trust before it is noticed.
- Production Quality Assurance Challenge. Distinguish production LLM quality assurance from traditional deterministic software quality assurance, including why monitoring alone may miss problems.
- Three Pillars of LLM Quality. Explain how prevention, detection, and correction work together to maintain LLM quality, and give an example of each as described in the section.
- Prompts as quality contracts. Explain why a production prompt must specify expected behavior rather than give a broad instruction, using the legal-document assistant’s contradictory answers as an example.
- Need for continuous quality assurance. Explain why LLM quality must be checked continuously rather than only before deployment.
- Golden dataset regression checks. Describe how a golden dataset detects regressions without requiring identical LLM answers.
- Per-response automated quality scoring. Explain how automated scoring supports real-time detection of quality changes.
- Shadow testing before deployment. Trace how shadow testing compares a candidate model or prompt with production while protecting users, and state its operational trade-off.
- Evidence for model switching. Describe the signals that warrant evaluating a model upgrade or switch and how the team should make that decision.
- Production observability gap. Explain why healthy-looking logs and good development outputs can still leave teams unable to diagnose confusing production responses, and how Langfuse addresses that gap.
- Traces and diagnostic signals. Describe the kinds of interactions and signals Langfuse brings together, including workflow traces, cost and latency, prompt or model versions, output quality, and failures.
What it builds on
- Evaluating agent performance (chapter 9) · Assumes the earlier chapter’s foundation for evaluating LLM and agent performance.
- Multimodel fallback: Matching each query to the right model (chapter 9) · Matching queries to models through multimodel fallback
- Token streaming: Presenting answers incrementally or at the same time (chapter 9) · Introduces response streaming as a way to present answers incrementally.
- LLM-as-a-judge: A holistic approach (chapter 9) · The earlier discussion of LLM-as-a-judge evaluation.
- Designing components of a prompt for reliability (chapter 2) · prompt components designed for reliability
- Identifying and measuring hallucinations (chapter 9) · Identifying and measuring problems in LLM outputs
- Model selection as a reliability decision (chapter 2) · Treating model selection as a reliability decision
Section by section
Introduction
This introduction explains why an LLM that succeeds in testing can still fail in production and why evaluation must be backed by operational infrastructure.
- Production variability of LLMs: Explain why an LLM that works well in development may fail under real-world conditions, and how those failures differ from consistent software bugs.
Why it matters: Passing internal tests does not guarantee that an LLM system will behave reliably for real users. - Evaluation and operational infrastructure: Explain why evaluation by itself cannot keep an LLM system running reliably and what kinds of infrastructure must act on its measurements.
Why it matters: Finding a failure is only useful in production if the surrounding system can detect and address problems as they arise.
Key terms: nondeterministic, production, context quality, evaluation, deployment architectures, monitoring systems, quality-assurance pipelines
10.1 Introducing large language model operations
This section defines LLMOps, explains why production reliability requires more than uptime, and introduces an architecture that uses monitoring to drive improvement.
- LLMOps production focus: Explain what LLMOps manages and how its focus differs from traditional MLOps.
Why it matters: A model that succeeds in staging may behave differently under real user queries. - Reliability beyond uptime: Justify why a production LLM system needs continuous evaluation of outputs alongside operational monitoring.
Why it matters: Healthy service metrics alone cannot show whether users are receiving trustworthy answers. - Production response processing layers: Describe how the architecture takes a user query through input processing, model execution, and output processing.
Why it matters: Separating these responsibilities makes clear that a reliable response requires more than a model call. - Monitoring improvement feedback loop: Trace how monitoring and observability feed continuous improvement in the LLMOps architecture.
Why it matters: Production observations are useful when they lead to changes that preserve or improve quality.
Key terms: LLMOps, MLOps, prompt management, RAG pipeline evaluation, latency, uptime, quality, safety, user trust, input processing
10.2 Serving LLMs: Hosted APIs vs. open source models
Introduces the choice between hosted APIs and self-hosted open source models as a foundational production decision with consequences beyond implementation.
- Production model hosting choice: Explain that choosing between a hosted API and running an open source model on your own infrastructure is a foundational production decision, not merely a coding choice. Describe how costs, provider outages, and compliance requirements can affect that decision.
Why it matters: A serving choice that looks simple in a demo can have major production consequences.
Key terms: hosted API, open source model, own infrastructure, compliance requirements
10.2.1 Using hosted APIs
This section explains why teams often start with hosted LLM APIs and what cost, data-handling, and model-control trade-offs emerge in production.
- Hosted API Starting Advantages: Explain why hosted APIs are attractive when building an LLM application, especially early in a project or with limited engineering resources.
Why it matters: The operational work a provider takes on is a major reason hosted APIs are an appealing starting point. - Usage Based Cost Growth: Explain why hosted-API costs estimated from short test prompts may rise sharply with real users and increasing traffic.
Why it matters: Testing with realistic request lengths helps reveal a production cost that a quick demo can hide. - Third Party Data Trade-Off: Describe the data-handling concern with hosted APIs and the compliance trade-off involved in running models locally instead.
Why it matters: Keeping data off a provider’s servers does not, by itself, remove the work of meeting data-protection obligations. - Hosted Model Control Limits: Explain how limited insight into hosted models and provider updates can affect production behavior.
Why it matters: An application can change behavior even when its own code has not changed.
Key terms: hosted APIs, fast time to market, zero infrastructure, per token, use-based pricing, input, output, third party, compliance, data segregation
10.2.2 The open source alternative
This section explains the control gained by self-hosting an open source LLM, the infrastructure that control requires, and why improving model quality makes the option increasingly practical.
- Self-hosting control tradeoff: Explain what teams gain by running an open source model in their own environment and why that control brings more operational responsibility.
Why it matters: The ownership benefits are meaningful only if your team can take on the work that comes with them. - Production serving requirements: Describe the main infrastructure and operational capabilities this section says a production self-hosted LLM needs.
Why it matters: A model that runs locally is not yet a production service. - Improving open source viability: State the section's qualified claim about the quality and cost of a well-tuned open source model compared with a hosted model.
Why it matters: Improving quality makes self-hosting worth considering despite its operational demands.
Key terms: open source models, self-hosting, serving stack, observability, GPU infrastructure, load balancing, model management, monitoring, scaling logic, well-tuned Mistral 7B
10.2.3 The hybrid solution: Best of both worlds
The section explains how a hybrid LLM service routes queries between local and hosted models, and why production routing must be more adaptable than keyword matching.
- Hybrid model routing: Explain how a hybrid service uses a small local model for common or short factual queries, sends nuanced or risky queries to a hosted LLM, and falls back to a hosted API when the local model fails or is overloaded.
Why it matters: Different queries and operating conditions call for different models rather than one model handling everything. - Production routing adaptability: Explain why keyword matching is brittle for production routing and describe the two alternatives given: semantic routing with embeddings and topic clusters, or a cheap LLM that classifies intent.
Why it matters: A router that depends on exact words can misroute users who phrase familiar requests differently.
Key terms: hybrid approach, local model, hosted LLM, fall back, keyword-matching, semantic routing, topic clusters, classify intent
10.3 Building LLM-native monitoring systems
A refund-policy failure shows why healthy service metrics are not enough to monitor an LLM that advises customers.
- Service health versus answer quality: Explain how a chatbot can appear healthy on a conventional dashboard while giving customers harmful, incorrect advice.
Why it matters: A chatbot can damage customer relationships even when its service dashboard shows no technical problem.
Key terms: traditional monitoring metrics, LLM monitoring
10.3.1 What really matters: The four questions
This section frames production LLM monitoring around four daily questions: speed and availability, answer quality, customer satisfaction, and cost.
- Four production monitoring questions: Explain the four questions a team should ask daily about an LLM service: can customers get help quickly, are answers useful, do customers leave satisfied, and are costs under control. Describe why technical performance alone cannot answer the other three questions.
Why it matters: A healthy server does not necessarily mean customers are getting useful help at a sustainable cost.
Key terms: response times, system availability, answer quality, user behavior, LLM costs
10.3.2 Logging what matters
This section shows how request logs that expose where an LLM application spends time and money lead to better optimization decisions.
- Request logs for diagnosis: Explain why LLM request logs need to show more than HTTP status codes and total response times, and how time and cost information helps a team investigate anomalies.
Why it matters: Useful request logs point the team toward the part of the system worth investigating. - Inference as latency bottleneck: Describe how the request timeline identifies LLM inference as the main source of latency and why the team addressed slow responses with streaming and model optimization rather than scaling search infrastructure.
Why it matters: A stage-by-stage timeline helps avoid optimizing a part of the request that is not causing most of the delay. - Token costs and priorities: Explain why the cost breakdown directs the team toward shorter outputs, more efficient prompts, and model routing instead of infrastructure changes when reducing spending.
Why it matters: A cost breakdown directs savings efforts toward the expense that actually dominates the bill.
Key terms: request logging, technical performance, business effect, LLM inference, response streaming, model optimization, token generation, output token length, prompt efficiency, intelligent model routing
10.3.3 Setting up alerts that help
This section shows how LLM alerting can reduce false positives by tracking meaningful patterns and efficiency instead of isolated threshold breaches.
- Contextual LLM alert signals: Explain why LLM alerts should focus on patterns and context rather than simple thresholds, using the section’s latency, error, and cost examples.
Why it matters: Alerts are more useful when they point to problems worth investigating rather than every temporary spike.
Key terms: absolute thresholds, tokens per second, errors cluster by user type or question topic, cost per token, baseline
10.3.4 Catching cost explosions before they hurt
The section shows how cost monitoring can catch different sources of rising LLM costs and make alerts useful for investigation.
- Cost anomalies and actionable alerts: Explain how prompt changes, model-routing bugs, and shifts in user questions can raise LLM costs, and why alerts that include likely causes help teams investigate faster.
Why it matters: Knowing what might drive a cost spike makes an alert useful for finding the problem, not just noticing the bill.
Key terms: cost monitoring, model mix, root-cause investigation
10.3.5 Building dashboards that drive action
This section shows how a production LLM dashboard helps a team spot problems, investigate alerts, and connect technical performance to customer and business outcomes.
- Actionable dashboard questions: Explain the three questions the dashboard answers: what is happening now, where to investigate, and what the business impact is.
Why it matters: The dashboard is useful when its information points the team toward a decision or a fix. - Business outcome metrics: Distinguish metrics tied to customer success and unit economics from isolated technical or aggregate metrics, using the section’s examples.
Why it matters: A low total cost or fast response alone does not show whether customers are getting useful help. - Alert investigation context: Describe the context shown when an alert fires and how it helps the team resolve problems.
Why it matters: An alert is more actionable when it also gives the team clues about what changed and where to look.
Key terms: real-time request volume, success rates, business impact, cost per satisfied customer, questions resolved without escalation, customer success, sustainable unit economics, recent deployments, correlated metrics, historical context
10.3.6 Output-quality monitoring
This section shows how to detect declining output quality through automated checks, statistical patterns, and customer-support signals.
- Layered Output Quality Checks: Explain how rule-based filters and an LLM-as-a-judge can help monitor outputs, while recognizing that quality is often subjective and context-dependent.
Why it matters: Different checks help surface poor outputs that might otherwise reach users unnoticed. - Statistical Quality Drift Signals: Describe the response patterns the section recommends tracking for signs of quality drift and what a change in those patterns might suggest.
Why it matters: Tracking patterns can reveal a deterioration that individual response checks may miss. - Customer Support Quality Signals: Describe how user behavior and answer consistency can reveal quality problems in a customer-support bot.
Why it matters: User-facing signals can expose unclear or inconsistent answers even when the bot continues to respond.
Key terms: automated content filters, rule-based systems, LLM-as-a-judge, statistical quality monitoring, quality drift, refusal-rate tracking, repetition detection, language consistency, user satisfaction scores, follow-up question rates
10.4 User experience and feedback monitoring
This section explains why production monitoring must look beyond technical performance to whether users find an AI system helpful.
- User-centered success monitoring: Explain why good uptime, response times, and costs do not establish that an LLM is successful, and why monitoring should include explicit feedback and implicit behavioral signals.
Why it matters: A healthy dashboard can hide an experience that drives users away.
Key terms: user satisfaction, technical metrics, explicit feedback, implicit behavioral signals
10.4.1 Explicit feedback collection
This section explains how voluntary feedback options capture different kinds of user input and how changes in feedback participation can warn of declining satisfaction.
- Voluntary feedback options: Explain why the team offered thumbs buttons, star ratings, and optional comments instead of requiring users to rate responses.
Why it matters: People can give feedback in the form that suits them without being pushed into a rating that does not reflect their experience. - Feedback participation trends: Explain why the team monitored feedback rates as well as the feedback users submitted.
Why it matters: A participation drop may give the team time to investigate before lower satisfaction appears in the scores.
Key terms: thumbs-up/thumbs-down, 1–5 star ratings, optional comment fields, forced feedback, feedback rates, participation rates, leading indicators, user satisfaction scores
10.4.2 Implicit feedback signals
This section shows how user behavior can provide implicit feedback about satisfaction with an AI system.
- Behavioral signals of satisfaction: Describe how session abandonment, immediate rephrasing, copying a response, and return use can suggest frustration, usefulness, or trust. Explain why several immediate follow-up questions may signal confusion rather than satisfaction.
Why it matters: These signals help you notice whether users are finding the AI useful even when they leave no rating. - Value of implicit feedback: Explain why observing behavior can be more revealing than relying only on explicit ratings.
Why it matters: Looking beyond ratings can reveal problems users might not report directly.
Key terms: implicit feedback, session abandonment, rapid query refinement, copy-paste behavior, return use patterns, session frequency, implicit signals, explicit feedback
10.4.3 Building actionable feedback loops
This section shows how to turn recurring user feedback into prioritized system changes and verify their effect on customer and business outcomes.
- Pattern-Based Feedback Triage: Explain how the team categorized and prioritized negative feedback, then looked for recurring patterns that suggested system changes rather than fixing complaints one at a time.
Why it matters: Patterns help the team choose changes that address more than one isolated complaint. - Feedback and Business Outcomes: Describe how the team checked whether AI interactions delivered value beyond satisfaction ratings, using task completion, support-ticket volume, and customer retention.
Why it matters: These measures show whether a better-rated AI experience also helps users and the business. - Billing Feedback Improvement Loop: Trace how billing escalations and user complaints led the team to identify a gap in complex-case handling, change routing, and measure the result.
Why it matters: The example shows a feedback loop ending in a targeted change and a measurable check on its effects.
Key terms: feedback triage, business effect, prompt adjustment, knowledge-base gap, task-completion tracking, support-ticket reduction, customer-retention correlation, billing complexity, specialized prompts, user-initiated billing escalations
10.5 Ensuring high-quality outputs in production
This section explains why production LLM output quality is difficult to assure: plausible-looking answers can be wrong, and failures are not necessarily consistent or obvious.
- Convincingly Wrong LLM Answers: Explain how an LLM answer can look authoritative while containing errors, and why this kind of failure can damage trust before it is noticed.
Why it matters: An answer that looks credible may be more harmful than an obvious failure because users may rely on it. - Production Quality Assurance Challenge: Distinguish production LLM quality assurance from traditional deterministic software quality assurance, including why monitoring alone may miss problems.
Why it matters: Production quality checks must account for failures that vary by context and may not show up as obvious system faults.
Key terms: convincingly wrong, user trust, traditional software quality-assurance, nondeterministic failures, context-dependent quality, gradual degradation
10.5.1 The three-pillar quality framework
This section defines prevention, detection, and correction as three complementary pillars for maintaining LLM output quality.
- Three Pillars of LLM Quality: Explain how prevention, detection, and correction work together to maintain LLM quality, and give an example of each as described in the section.
Why it matters: The three pillars distinguish avoiding quality problems from spotting and addressing the ones that still occur.
Key terms: prevention, detection, correction, LLM quality
10.5.2 Prompt engineering for consistent quality
This section explains how specific prompts act as quality contracts for consistent LLM behavior in production.
- Prompts as quality contracts: Explain why a production prompt must specify expected behavior rather than give a broad instruction, using the legal-document assistant’s contradictory answers as an example.
Why it matters: A prompt that sounds clear to a person may still leave the model room to give inconsistent answers. - Production prompt quality requirements: Describe the qualities production prompts should support beyond performance: predictable responses across diverse inputs, quality under load, and escalation when the model reaches its limits.
Why it matters: A prompt must remain dependable across many users and situations, not just perform well on a single example.
Key terms: quality contracts, quality-control mechanisms, ambiguity, consistency, reliability, predictable outputs, escalation paths
10.5.3 Continuous quality monitoring with automated testing
This section explains how continuous tests, response scoring, and shadow testing detect LLM quality regressions and inform model-switching decisions.
- Need for continuous quality assurance: Explain why LLM quality must be checked continuously rather than only before deployment.
Why it matters: A bot that worked well at launch can deteriorate quietly while users continue to rely on it. - Golden dataset regression checks: Describe how a golden dataset detects regressions without requiring identical LLM answers.
Why it matters: This catches meaningful changes in variable answers without treating every wording difference as a failure. - Per-response automated quality scoring: Explain how automated scoring supports real-time detection of quality changes.
Why it matters: Per-response checks can surface emerging quality problems while the system is serving users. - Shadow testing before deployment: Trace how shadow testing compares a candidate model or prompt with production while protecting users, and state its operational trade-off.
Why it matters: Shadow tests provide evidence from real traffic before users are exposed to a proposed change. - Evidence for model switching: Describe the signals that warrant evaluating a model upgrade or switch and how the team should make that decision.
Why it matters: Evidence-based reassessment helps avoid both staying with a failing model and switching on guesswork.
Key terms: quality assurance, drift, quality regressions, golden dataset, known-good response characteristics, automated quality scoring, real-time quality metrics, alerts, shadow testing, candidate models
10.6 Observability in practice: Introducing Langfuse
The section introduces Langfuse as a way to observe LLM behavior in production and uses Huntr’s résumé builder to show how that visibility supports debugging and improvement.
- Production observability gap: Explain why healthy-looking logs and good development outputs can still leave teams unable to diagnose confusing production responses, and how Langfuse addresses that gap.
Why it matters: A system can appear healthy while its answers fail users, so teams need visibility into the LLM workflow itself. - Traces and diagnostic signals: Describe the kinds of interactions and signals Langfuse brings together, including workflow traces, cost and latency, prompt or model versions, output quality, and failures.
Why it matters: Connecting quality and operational signals to individual interactions makes production problems easier to investigate. - Huntr résumé builder example: Describe how Huntr used Langfuse to track and evaluate résumé generations, improve prompts, and detect regressions before confidently releasing changes.
Why it matters: Huntr illustrates how observability can turn prompt releases from guesswork into changes whose effects can be checked.
Key terms: Langfuse, observability, production, end-to-end interactions, token counts, latency, prompt versioning, output-quality scores, trace-level details, Huntr
10.7 Summary
Where this is used later
- The six principles of reliable AI (chapter 11) · production monitoring of LLM systems
Concepts explained in this chapter
- costThe monetary expense of using an API service, typically based on usage.
- monitoringThe ongoing observation of a deployed AI system's behavior, performance, and quality.
- LLMOpsLLMOps is the engineering discipline of managing large language models in production, including their deployment, evaluation, and ongoing operation.
- hybrid deploymentA hybrid deployment architecture combines multiple deployment approaches or environments within one system.
- degradationA decline in the quality of a system's outputs compared with an earlier or expected level.
- feedback loopA process that uses information about a system's outputs to guide its later behavior or improvement.
- evaluation datasetA collection of examples used to assess a model or system's behavior against defined criteria.
- shadow testingShadow testing evaluates a system or change alongside a live system without allowing the test results to affect user-facing behavior.
- observability systemA software platform that collects and presents information about system behavior to support analysis and troubleshooting.
- managed APIA hosted API is an application programming interface provided by an external service whose infrastructure runs the underlying model or software.
- prompt engineeringThe deliberate design of instructions and context to guide a language model's behavior and outputs.
- open-source modelA model whose weights are made available under terms that permit users to obtain and run it, and potentially modify or adapt it.
- customer satisfactionThe degree to which users are satisfied with a system or its outcomes.
- consistencyThe degree to which a language model produces stable and coherent outputs across responses.
- promptInput instructions or context supplied to an AI model to guide its response.
- implicit feedbackImplicit feedback is evaluative information inferred from user behavior rather than directly supplied by users.
- model selectionThe process of choosing an AI model for a particular application or task.
- continuous improvementAn ongoing process of using system information to make iterative changes that improve performance or quality.
- deploymentThe release of software or a software change into an environment where it can operate.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.