Chapter 2: Generating trustworthy responses with prompt engineering

Introduction

The introduction explains how prompt engineering guides LLM outputs and why plausible hallucinations make reliable prompting important.

• Prompt Engineering for Reliability: Explain how prompts guide LLM generation and why prompt engineering is iterative. Describe how prompt quality affects the reliability of outputs.
  Why it matters: Prompt design is a practical way to influence whether an LLM produces a dependable answer.
• Plausible Hallucination Risk: Describe what counts as an LLM hallucination here and why it can be particularly dangerous.
  Why it matters: An answer that sounds convincing can still be false or unsupported.

Key terms: prompt engineering, context, instructions, examples, hallucination, provided context

2.1 Choosing the right model

This section establishes model selection as the first and most consequential reliability decision before parameter tuning or prompt design.

• Model selection as a foundation: Explain why selecting a model comes before tuning parameters or crafting prompts: the choice sets limits on cost, latency, capability, and available parameters, and different model categories suit different reliability requirements.
  Why it matters: Choosing a model first helps you avoid trying to solve a model-level limitation with settings or prompts.

Key terms: model selection, cost ceiling, latency floor, capability range, available parameters, reliability requirements

2.1.1 Reasoning vs. nonreasoning models

This section contrasts reasoning and nonreasoning models and gives a practical rule for choosing between them.

• Reasoning model tradeoffs: Explain why reasoning models can be useful for complex multistep tasks and what they cost compared with nonreasoning models.
  Why it matters: The extra reasoning capability is valuable only when the task warrants its latency and cost.
• Reasoning model parameter support: Describe why sampling settings must be checked before using them with a reasoning model.
  Why it matters: A setting cannot be relied on to control output unless the chosen model supports it.
• Practical model selection rule: Apply the section’s rule of thumb for when to start with a nonreasoning model, try a reasoning model, or choose a smaller model.
  Why it matters: This rule avoids paying for extra capability when a simpler model already performs well enough.

Key terms: reasoning models, nonreasoning models, multistep problems, sampling parameters, temperature, top-p, reasoning_effort, non-reasoning model, reasoning model, quality bar

2.1.2 The cost-capability spectrum

This section explains how to balance model capability, cost, and operational control when choosing between model tiers and commercial APIs versus open source models.

• Model tier and task fit: Explain why smaller, cheaper models suit high-volume routine tasks while larger flagship models suit harder, quality-sensitive tasks.
  Why it matters: Matching the model to the task helps control cost without treating every request as equally difficult.
• Open source deployment tradeoffs: Describe the reliability benefits of open source models and the operational cost of self-hosting them compared with commercial APIs.
  Why it matters: The deployment choice affects reliability and privacy as well as the infrastructure a team must operate.

Key terms: flagship model, mini variant, high-volume tasks, quality-sensitive tasks, open source models, commercial APIs, model life cycle, self-hosting, fine-tuning

2.1.3 Model selection as a reliability decision

Model selection should remain flexible because available models and their prices change.

• Configurable model selection: Explain why a production system should keep its model name in configuration rather than hardcode it throughout the codebase.
  Why it matters: Keeping the model configurable lets you respond to deprecations and better options without changing model names across your codebase.

Key terms: model selection, configuration parameter, deprecated

2.2 Tailoring LLM settings for maximum reliability

This section introduces configurable LLM settings as a way to steer responses toward greater reliability for a given task.

• Settings and response reliability: Explain that tuning LLM settings can influence generated responses and help make them more accurate, consistent, and relevant to a task while reducing hallucination risk.
  Why it matters: The settings you choose can affect how dependable the model’s answers are for your application.

Key terms: temperature, top-p sampling, maximum output length, stop sequences, penalty strengths, hallucinations

2.2.1 Optimizing temperature for predictable outputs

This section explains how temperature changes response variability, why predictability does not guarantee accuracy, and how to choose settings for a chatbot.

• Temperature and token probabilities: Explain how temperature changes next-token probabilities and the resulting variability of chatbot responses.
  Why it matters: Temperature is a way to control how consistent or varied a chatbot sounds.
• Predictability and factual accuracy: Distinguish consistent output from factual accuracy, including the limits of temperature zero.
  Why it matters: A stable answer still needs to be checked for truth.
• Use-case temperature balance: Describe how to choose and test temperatures for focused answers versus varied conversation, and identify the risk of excessively high settings.
  Why it matters: Testing settings against the task helps avoid replies that are either too rigid or too erratic.
• Model-specific temperature support: State that temperature availability and range depend on the model, and describe where it can be configured when supported.
  Why it matters: A chosen temperature is useful only if the model supports it and accepts that value.

Key terms: temperature, next-token probabilities, deterministic, determinism, factual accuracy, hallucinations, temperature=0, product inquiries, diversity, incoherence

2.2.2 Top-P Sampling technique

This section explains how top-p sampling limits candidate tokens and how to tune it to balance response variety with reliability.

• Top-p candidate token selection: Explain how top-p sampling selects candidate tokens at each generation step using a cumulative probability threshold.
  Why it matters: The threshold explains why changing top-p changes which words the model can consider.
• Response diversity and reliability tradeoff: Distinguish the effects of lower and higher top-p values on focus, detail, creativity, and the risk of irrelevant or incorrect information.
  Why it matters: The tradeoff helps you choose between a tightly focused answer and a richer one without assuming either is always better.
• Task-based top-p tuning: Describe how to adjust top-p based on observed output quality and the task’s need for variety versus reliability, while avoiding simultaneous changes to temperature and top-p.
  Why it matters: Task-aware tuning avoids treating one top-p setting as best for every application.

Key terms: top-p, nucleus sampling, candidate tokens, cumulative probability threshold, lower top-p, higher top-p, consistency, creativity, hallucinations, top-p value

2.3 Limiting output and turns to reduce hallucination risk

The section explains how scoping response length and conversation turns can reduce opportunities for unsupported claims without cutting off useful information.

• Scoped answers and hallucination risk: Explain why unnecessary elaboration can increase hallucination risk, while distinguishing reliably scoped answers from answers that are merely short.
  Why it matters: A concise answer helps only if it still answers the user’s question adequately.
• Response length controls: Describe how maximum token limits and stop sequences constrain a generated response.
  Why it matters: These controls give developers practical ways to prevent a response from wandering beyond the question.
• Conversation turn controls: Describe how ending rules and user controls can keep a conversation from meandering.
  Why it matters: Reliability depends on controlling unnecessary continuation across turns as well as within a single answer.

Key terms: unsupported claims, output length, hallucinations, chain-of-thought prompting, maximum token limits, stop sequences, LLM API, postprocessing, maximum number of turns, terminating the interaction

2.4 Applying frequency and presence penalties for balanced content

This section explains how frequency and presence penalties discourage repetition, how they differ, and how they can make chatbot responses more varied.

• Frequency and presence penalties: Distinguish the two penalties: frequency penalty grows as a token appears more often, while presence penalty discourages reuse once a token has appeared, regardless of how often.
  Why it matters: The distinction helps you choose which kind of repetition you want to discourage.
• Repetition in chatbot conversations: Explain how discouraging repeated words, suggestions, and topics can make a customer-service chatbot’s replies more varied and its conversation less stale.
  Why it matters: A varied reply is more useful than repeatedly recommending the same product or reiterating the same point.
• Penalty value guidance: State how higher penalty values affect token sampling and give the range and usual values specified in the section.
  Why it matters: The values connect the idea of discouraging repetition to a setting you can adjust.

Key terms: frequency penalty, presence penalty, token, repetitive or redundant outputs, frequency penalties, presence penalties, penalty values, sampling new tokens

2.5 Minimizing intrinsic randomness for stable performance

The section explains why LLM outputs can vary despite low-randomness settings and describes ways to make responses more stable without always eliminating useful diversity.

• Intrinsic output variability: Explain why repeated responses to the same prompt may differ even when sampling settings minimize randomness, using the product-recommendation example.
  Why it matters: It explains why low-randomness settings alone do not guarantee identical responses.
• Fixed seed reproducibility: Describe how setting the same random seed can make repeated generations more reproducible when the prompt and other parameters stay the same.
  Why it matters: A fixed seed is a practical way to improve repeatability when comparing or relying on generated responses.
• Multiple output aggregation: Explain how generating several responses to one prompt and selecting or combining common results can reduce run-to-run variation.
  Why it matters: Combining several runs offers another route to stable recommendations when individual runs vary.
• Consistency and diversity balance: Justify why an application might retain some output variability instead of aiming for complete determinism.
  Why it matters: Stability is a design choice to balance against the value of varied interactions.

Key terms: intrinsic randomness, temperature, top-p sampling, token probabilities, fixed random seed, seed parameter, reproducible, majority voting, weighted averaging, ensemble averaging

2.6 Foundations of prompt engineering for reliable LLMs

This section introduces effective prompts as a way to align LLM behavior with reliability goals and reduce hallucinations.

• Prompts and reliable behavior: Explain why crafting effective prompts matters for reliable LLM outputs: prompts help align model behavior with reliability goals and reduce hallucinations.
  Why it matters: A prompt is part of making an LLM's responses more dependable, not just telling it what task to do.

Key terms: effective prompts, reliability goals, hallucinations

2.6.1 Designing components of a prompt for reliability

This section shows how to structure a prompt so the model has a clear task, relevant evidence, predictable output requirements, and instructions against unsupported claims.

• Clear task and grounded input: Explain how clear instructions, relevant context, and focused input data help keep a response tied to the intended task and the information provided.
  Why it matters: A clear task and focused evidence give the model less room to drift into irrelevant or unsupported content.
• Output constraints and quality criteria: Describe how specifying the response format and explicit quality criteria constrains a model’s answer, using the product-review summary prompt as an example.
  Why it matters: A specified structure and an explicit ban on unsupported claims help keep answers predictable and grounded.

Key terms: clear instructions, relevant context, focused input data, expected output format, explicit quality criteria, factual accuracy, unsupported claims

2.6.2 Crafting basic prompts with a focus on dependability

This section shows how specific instructions and deliberate prompt components make LLM responses more targeted and dependable.

• Specific instructions for targeted responses: Explain why a vague prompt can produce a reasonable but unfocused response, and how adding a clear constraint changes the result in the ocean example.
  Why it matters: A small change in instructions can prevent an otherwise reasonable answer from missing the intended form.
• Components of a dependable prompt: Describe how clear instructions, relevant context, output expectations, and variable customer input each guide a chatbot’s response.
  Why it matters: These components help a chatbot answer the customer’s actual query in the intended domain and style.

Key terms: clarity, specificity, instructions, targeted response, clear instructions, relevant context, structured output, variable input, customer query

2.7 Prompt engineering techniques for preventing hallucinations

The section explains why instructions alone can be insufficient and introduces prompting as a foundation that complements retrieval.

• Limits of instructions alone: Explain that models can handle basic tasks from instructions alone, but clear examples and reasoning steps can help them perform more specialized tasks.
  Why it matters: This explains why prompt design can matter even when a model already follows instructions.
• Complementary prompting and retrieval: Describe how prompt engineering and retrieval complement each other: prompting develops model capabilities, while retrieval connects models to extensive real-time knowledge.
  Why it matters: It places prompt design alongside access to external information rather than treating either as a complete solution.

Key terms: instructions, clear examples, reasoning steps, prompt design, prompting, retrieval, RAG, real-time knowledge

2.7.1 Zero-shot prompting

This section explains what zero-shot prompting can do, where it falls short, and why prompts must be tested on the model that will use them.

• Zero-shot instructions and capabilities: Explain that zero-shot prompting gives a model an instruction without example demonstrations, though it may include context. Describe how prior training can let the model complete a familiar task such as classifying a negative movie review.
  Why it matters: It helps you recognize when an instruction alone may be enough.
• Zero-shot reliability limits: Explain why zero-shot responses can be generic or inconsistent when a task needs company-specific information or is complex, and describe the section’s suggested role for context or examples.
  Why it matters: A plausible generic answer may still fail to address the user’s actual situation.
• Target-model prompt testing: Explain why a prompt should be tested on its target model and may need adjustment when the model or its version changes.
  Why it matters: Testing on the intended model helps catch differences that a successful prompt elsewhere would miss.

Key terms: zero-shot prompting, example demonstrations, pretraining, post-training alignment, zero-shot, return policy, context, examples, target model, wording

2.7.2 Few-shot prompting for contextual stability

This section shows how examples in a prompt help an LLM follow a desired pattern and interpret unfamiliar terms, while showing that examples alone may not make multistep reasoning reliable.

• Examples for contextual stability: Explain how few-shot prompting uses example inputs and outputs to make responses more consistent in style, format, or interpretation. Describe how the model applies an example pattern to a new request, such as using “cozzy” to recommend a sweater.
  Why it matters: Well-chosen examples can make a chatbot’s responses more consistent and help it interpret terms specific to its context.
• Limits on multistep reasoning: Explain why a few examples do not guarantee correct answers on complex reasoning tasks. Use the odd-number example to identify the model’s wrong answer and the calculation that corrects it.
  Why it matters: A response that matches the examples’ format can still be wrong when the task requires calculation or several reasoning steps.

Key terms: few-shot prompting, example demonstrations, one-shot generalization, multistep logical reasoning, chain-of-thought prompting

2.7.3 Chain-of-thought prompting for transparent reasoning

This section explains how showing intermediate reasoning steps can help an LLM handle multistep tasks, including when a prompt has no worked example.

• Chain-of-thought reasoning examples: Explain how chain-of-thought prompting uses a worked example with intermediate steps to help a model follow multistep logic. Illustrate this with the section’s task of adding only the odd numbers before deciding whether their sum is even.
  Why it matters: A worked chain makes the logic of an answer explicit rather than showing only the final result.
• Zero-shot step-by-step prompting: Distinguish the section’s zero-shot CoT variation from a worked-example prompt, and describe what the phrase “Let’s think step by step” elicits in the apples example.
  Why it matters: The section shows that prompting for visible steps does not always require a worked demonstration.
• Few-shot logical reasoning limit: Explain why the section contrasts simple few-shot product association with an order-discount decision, and state how CoT is proposed to address the latter.
  Why it matters: Getting a familiar-looking example right does not guarantee that a chatbot will apply a logical rule correctly.

Key terms: chain-of-thought (CoT) prompting, intermediate reasoning steps, reasoning chains, few-shot examples, zero-shot variation, Let’s think step by step, few-shot learning, logical conditions, CoT prompting

2.7.4 Enhanced CoT prompt

This section shows how step-by-step chain-of-thought prompts can improve a chatbot’s reasoning and when explicit instructions may be useful.

• Stepwise discount reasoning: Explain how the discount example applies chain-of-thought prompting: check the order number’s last digit, determine whether it is odd, and then decide eligibility. Describe how prompting for intermediate steps helps the model avoid illogical leaps.
  Why it matters: Intermediate steps make it easier to get and inspect a logically supported answer.

Key terms: CoT prompting, intermediate steps, chain of thought

2.7.5 Automatic CoT for scalable reasoning

This section explains how Auto-CoT scales step-by-step reasoning through generated or learned intermediate steps, while warning that those steps need validation.

• Input-specific reasoning chains: Explain why Auto-CoT replaces manually written reasoning chains and how a separate reasoner model can generate steps for the main prompt. Describe the legal-assistant example as a progression from identifying issues and relevant law to reasoning about claims and offering advice.
  Why it matters: Input-specific steps can make detailed reasoning practical across many different user questions.
• Fine-tuned intermediate reasoning: Describe the alternative Auto-CoT approach of fine-tuning the main model on diverse CoT examples so it can generate intermediate steps for new problems.
  Why it matters: It distinguishes two ways to automate reasoning steps without confusing their roles.
• Generated chain validation: Justify validating automatically generated reasoning chains despite Auto-CoT’s scalability benefits.
  Why it matters: Automating reasoning does not remove the need to check whether that reasoning is trustworthy.

Key terms: Automatic chain of thought, Auto-CoT, reasoner model, main prompt, fine-tune, main model, CoT examples, intermediate steps, validate, generated chains of thought

2.7.6 Self-consistency for cross-verification

This section explains how comparing multiple responses to the same input can reveal inconsistencies and help produce a more reliable answer.

• Self-consistency sampling and signals: Explain how self-consistency samples several outputs for the same input and uses agreement or discrepancies to assess reliability.
  Why it matters: Comparing responses gives you a way to spot uncertainty that one plausible-sounding answer might hide.
• Inconsistent claim filtering: Describe how the recipe chatbot uses common themes across responses to filter a conflicting claim and present a synthesized answer, including the level of detail at which comparisons can be made.
  Why it matters: A response can sound plausible overall while containing a detail that conflicts with the user’s request.

Key terms: self-consistency, random seeds, prompt variations, cross-verifying, common themes, inconsistent or potentially hallucinated claims, synthesized output

2.7.7 Tree-of-thought prompting for structured decision-making

This section explains how tree-of-thought prompting compares possible solutions before producing a focused response to a complex query.

• Tree-of-thought branch evaluation: Explain why tree-of-thought prompting explores multiple reasoning branches instead of committing to one path, and describe how it evaluates and selects a solution.
  Why it matters: Comparing plausible paths can help the model avoid overlooking a better response.
• Customer-specific response selection: Apply the section’s ToT pattern to the damaged-TV query by describing plausible options, what customer details affect the choice, and what the customer sees in the final answer.
  Why it matters: The example shows how considering several options can still lead to a clear, useful customer-facing reply.

Key terms: tree-of-thought (ToT) prompting, decision tree, thought branches, return within 30-day policy, warranty claim, exception process, internal decision-making process

2.8 Project: Creating a reliable weather assistant with OpenAI’s function calls

The section explains how a weather assistant can use function calls to ground its answers in current external data instead of relying on the model’s training data.

• Grounded Weather Assistant Roles: Explain why the weather assistant uses the model to understand queries and compose answers but relies on a trusted external API for weather information.
  Why it matters: This division of work helps prevent a convincing but outdated or invented weather report.
• Function Call Response Flow: Trace the sequence from the model requesting weather data to its final response.
  Why it matters: Keeping the request, execution, and answer stages distinct makes clear where the answer’s factual basis comes from.
• Function Parameter Formatting: State what function calling does to the format of generated function parameters.
  Why it matters: A function request must match the function’s required parameter format to be usable.

Key terms: function calling, grounding, trusted external API, real-time data, function invocation, function results, final response, required parameters

2.8.1 Building a weather assistant

The section explains that the weather assistant answers current-weather questions by using a function to fetch live data from an external weather API.

• Live weather data retrieval: Explain how the weather bot obtains the information it needs to answer a current-weather question.
  Why it matters: A current-weather answer needs up-to-date information from outside the bot.

Key terms: weather bot, real-time weather data, function, external weather API

2.8.2 Getting started and defining the weather function

This section identifies the setup requirements and introduces a location-based weather function and its JSON description for the language model.

• Weather function and JSON schema: Describe the weather function’s input and output, and explain why a JSON schema is defined alongside it.
  Why it matters: This establishes what the weather function does and how it is described to the model.
• Weather assistant setup requirements: State the prerequisites named before implementing the weather function.
  Why it matters: These are the requirements the section names before starting the implementation.

Key terms: weather function, location, weather report, JSON schema, Python 3.7 or later, OpenAI API key, Weather API key, required packages

2.8.3 Integrating the chatbot

This section uses a weather chatbot to show how function calls bring external data into natural-language answers and reduce hallucination risk.

• Weather chatbot function-call flow: Trace how the weather assistant handles a user query: it invokes the weather function for the requested location and returns a natural-language answer based on the result.
  Why it matters: The example makes the connection between a user’s request, a function call, and the final answer concrete.
• Model and API responsibilities: Explain why the assistant delegates weather facts to an external API while the model interprets the question and writes the response.
  Why it matters: Separating language generation from factual retrieval helps explain why function calls can make answers more trustworthy.
• Authoritative sources beyond weather: Describe how the same function-call pattern can apply beyond weather by pairing an authoritative, up-to-date data source with a clear function interface the model can invoke for relevant queries.
  Why it matters: The weather example illustrates a reusable approach for questions that need current facts.

Key terms: function calls, get_weather, weather assistant, trusted external API, factual information retrieval, hallucinations, authoritative data sources, clear function interfaces, up-to-date information

2.9 Summary