liveBook · Manning

Zero-shot prompting

As explained in LLMs in Production

Zero-shot prompting is prompting a model to perform a task without providing task-specific examples in the prompt.

Zero-shot prompting asks a language model to perform a task without task-specific examples in the prompt. It is useful when examples or labeled data are unavailable, but it generally behaves less consistently than one-shot or few-shot prompting. section 7.1.3

A minimal zero-shot template

A simple template such as “Q: [User’s Prompt] A:” primes the model to treat the input as a question and answer it without demonstrations. Appending “think step by step” can encourage chain-of-thought behavior, improving multi-step math reasoning while producing longer responses and sometimes delaying the stopping point. section 7.1.3

Zero-shot prompting works best as a baseline rather than a guarantee of reliable reasoning. In one evaluation, a zero-shot chain-of-thought prompt raised Llama 3’s GSM8K accuracy to 14.5%, compared with 1%–5% from sending the questions alone; an optimized few-shot prompt reached 74.5%. section 12.2.5

Listing 12.3 DSPy for math
from transformers import AutoModelForCausalLM, AutoTokenizer
from transformers import BitsAndBytesConfig
import torch
import dspy
from dspy.datasets.gsm8k import GSM8K, gsm8k_metric
from dsp.modules.lm import LM
from dspy.evaluate import Evaluate
from dspy.teleprompt import BootstrapFewShot

model_name = "meta-llama/Meta-Llama-3-8B-Instruct"
quantization_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_use_double_quant=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
)


model = AutoModelForCausalLM.from_pretrained(
    model_name, 
    device_map="auto", 
    quantization_config=quantization_config,
    attn_implementation="sdpa",
)
tokenizer = AutoTokenizer.from_pretrained(model_name, use_fast=True,)

gms8k = GSM8K()
gsm8k_trainset, gsm8k_devset = gms8k.train[:30], gms8k.dev[:100]

Use the listing to inspect a complete DSPy evaluation flow: load a quantized Llama 3 model, define a zero-shot signature, evaluate it on GSM8K, and compare it with an optimized chain-of-thought program. section 12.2.5

How LLMs in Production covers it

How deep it goes
Explains templates, chain-of-thought prompting, response-length costs, and measured baselines, but defers reliable optimization to few-shot methods.
Examples
Demonstrates a Q/A template, “think step by step,” and a DSPy GSM8K evaluation using a Llama 3 model.
What it assumes
Assumes familiarity with prompting and language-model evaluation while distinguishing zero-shot, one-shot, few-shot, and chain-of-thought approaches.
What you can do after
Evaluate via DSPy by defining a zero-shot signature, running a GSM8K baseline, and comparing its accuracy with an optimized prompt.
How it connects
Connects zero-shot prompting with few-shot prompting, one-shot prompting, chain-of-thought prompting, prompt optimization, and hallucination evaluation.
Cover of LLMs in Production
liveBookmanning.com
Read LLMs in Production on liveBook Get the book at manning.com

Also explained in

Compare how 9 books explain zero-shot prompting →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLMs in Production. The section links point to the parts of the book it draws on.