Self-consistency

As explained in AI Agents in Action

Self-consistency is a reasoning technique that samples multiple candidate reasoning paths and selects an answer supported by their agreement.

Self-consistency generates several candidate reasoning paths or solutions for one problem, evaluates their agreement, and accepts the most frequent or most similar answer. It improves evaluation without requiring a low temperature; varied outputs can reveal which solution is most reliable. section 10.3.1

A practical self-consistency workflow:

• Prompt the model several times for the same problem, using reasoning examples when needed. section 10.3.1
• Collect the resulting plans or answers through repeated or batch processing. section 10.3.1
• Compare the outputs and select the answer with the strongest consistency or similarity. section 10.3.1

Time-travel reasoning with Prompt flow

A Prompt flow DAG processes a complex time-travel statement three times, using a few-shot Chain-of-Thought template. Batch execution collects the outputs, then an evaluation tool computes a mean embedding, ranks each answer by cosine similarity, and returns the most similar result. section 10.3.1

Figure 10.7 The self-consistency prompt generation beside the evaluation flow

Shows the generation flow beside the evaluation flow, making the separation between producing candidates and selecting an answer easy to inspect. section 10.3.1

How AI Agents in Action covers it

How deep it goes

Explains generation, batching, aggregation, and similarity ranking, while stopping short of broader reliability guarantees.

Examples

Demonstrates time-travel problems, few-shot prompts, Prompt flow DAGs, JSONL batch runs, embeddings, and cosine similarity.

What it assumes

Uses prompt engineering and Chain-of-Thought examples as the starting point for generating candidate solutions.

What you can do after

Operate/configure Prompt flow to batch-generate candidate answers and select one using an evaluation tool.

How it connects

Connects self-consistency with Chain-of-Thought prompting, tree-of-thought prompting, prompt chaining, and evaluation.

liveBookmanning.com

Read AI Agents in Action on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Introduction to Generative AI, Second EditionSelf-consistency is an evaluation technique that compares multiple independently generated answers to the same input to estimate answer reliability.
• 

  liveBookmanning.com
  Build a Reasoning Model (From Scratch)An inference technique that generates multiple candidate responses and selects or combines them to improve answer quality.
• 

  liveBookmanning.com
  Generative AI in ActionA self-consistency check compares multiple generated results or reasoning paths to identify inconsistent or unreliable outputs.
• 

  liveBookmanning.com
  The Art of AI Product DevelopmentSelf-consistency is a prompting technique that samples multiple reasoning or response paths and selects or synthesizes the answer supported by the most consistent results.
• 

  liveBookmanning.com
  Prompt Engineering in PracticeSelf-consistency is a sampling technique that generates multiple reasoning paths and selects or aggregates the conclusion that is most consistent across them.

Compare how 6 books explain self-consistency →

Related concepts

• chain-of-thought promptinguses · Chain-of-thought prompting is a technique that elicits or guides a model to produce or follow intermediate reasoning steps before a final answer.
• hallucinationaddresses · A hallucination is model-generated content that is false, fabricated, or unsupported by relevant evidence but presented as plausible or factual.
• modeuses · The statistical mode is the value or category occurring most frequently in a set of observations or outcomes.
• best-of-Nalternative to · Best-of-N is a technique that generates or evaluates multiple candidates and selects one according to a scoring or quality criterion.
• samplingused by · Sampling is a procedure for selecting or generating observations, values, or outcomes from a larger population, signal, or probability distribution according to a specified method.
• language modeluses · A language model is a computational model that represents patterns in linguistic tokens to estimate sequence probabilities and predict or generate language from context.

This explainer was generated by Manning's knowledge graph from the text of AI Agents in Action. The section links point to the parts of the book it draws on.