liveBook · Manning

Chain-of-thought prompting

As explained in Build an Advanced RAG Application (From Scratch)

Chain-of-thought prompting is a prompting technique that encourages a language model to produce intermediate reasoning steps before an answer.

Chain-of-thought (CoT) prompting guides a language model to work through intermediate reasoning before delivering an answer. It is useful when a direct instruction is insufficient for a complex or nuanced request, because the prompt asks for an ordered analysis rather than only a final result. section 5.3.2 section 5.5

A simple step-by-step instruction can shift a response from answer extraction to reasoning extraction. In a word problem, this means requesting the steps used to reach the numerical answer; adding “Let’s think step by step” to few-shot prompts is presented as a way to encourage that process. section 5.3.2

Use CoT prompts to make an analysis task explicit:

  • State the decision to make and the criteria to consider. section 5.3.2
  • Ask for comparisons, pros and cons, and recurring patterns in the supplied evidence. section 5.3.2
  • Require support from the provided results, then request a concise recommendation or summary. section 5.3.2

Hotel-result analysis

Given hotel search results for “Hotel near the Louvre with great food nearby,” a CoT prompt can ask the model to select the best match, compare the top three hotels, quote review evidence, identify review patterns, summarize strengths and weaknesses, and recommend an option. This turns retrieved results into a structured evaluation instead of an unprocessed list. section 5.3.2 section 5.5

def analyze_hotel_recommendations(query, results, model="gpt-4o"):
 
  
  prompt = f""" #A
  
  Analyze the following hotel search results and answer these questions using a chain of thought:
 
  1. Which hotel best matches the user's query: '{query}'?  Consider location, food options, and overall experience.   #B
  2. What are the pros and cons of the top 3 hotels?  Support your answer with quotes from the reviews.
  3. Are there any recurring themes or patterns in the positive/negative reviews?
  4. Summarize the top 3 hotels in bullet points highlighting their strengths and weaknesses.
  5. Based on the provided data, provide an overall recommendation to the user.
 
  Hotel Search Results:
  {results} #C
  """
  
 
 
 
 
 
 
 
  # Use OpenAI API to get the analysis
  chat_completion = client.chat.completions.create(
        messages=[
            {"role": "user", "content": prompt}
        ],
        model=model,
        temperature=0.7,
        max_tokens=1500,
    )  
  
  analysis = chat_completion.choices[0].message.content
  return analysis
 
analysis = analyze_hotel_recommendations(hotel_query, result) #D
display(Markdown(analysis))

Inspect a Python prompt that combines a user query and retrieved hotel results with explicit analysis steps and a final recommendation request. section 5.3.2

CoT is one prompting option alongside zero-shot and few-shot prompting. Prompt design affects output quality and relevance, while model and deployment choices still involve performance, cost, privacy, customization, and operational trade-offs. section 5.0 section 5.7

How Build an Advanced RAG Application (From Scratch) covers it

How deep it goes
Explains CoT through step-by-step word-problem reasoning and a Python hotel-analysis prompt, without detailing model internals.
Examples
Uses word problems and Louvre-area hotel reviews to contrast explained reasoning with direct answers.
What it assumes
Introduces CoT as one of three prompting paradigms while assuming familiarity with language-model instructions and queries.
What you can do after
Apply it via an OpenAI API prompt to compare search results, identify review patterns, and produce a recommendation.
How it connects
Connects CoT prompting with prompt engineering, zero-shot prompting, few-shot prompting, decoder models, and retrieval-augmented search.

Also explained in

Compare how 29 books explain chain-of-thought prompting →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build an Advanced RAG Application (From Scratch). The section links point to the parts of the book it draws on.