A translation app, a tutor, and a cooking assistant can use the same general-purpose language model, but they need different instructions. A prompt is the input given to a model. Prompt engineering goes beyond writing one good request: it means designing, templating, refining, and implementing prompts so an application produces useful, consistent responses for different users. section 7.0 section 7.1
Prompts guide behavior at use time rather than requiring expensive retraining. Wording matters: a small change can produce a very different answer. Start by stating the task and desired format explicitly; provide an example of the output when that helps. A request to translate English into French or summarize an article is a prompt, but an application also needs to handle requests from users who cannot—or will not—prompt it skillfully. section 5.2.3 section 7.2
Practical checks before settling on a prompt:
- Test for the answer format users expect, not merely an answer you personally like. One successful response is not evidence of a dependable user experience. section 7.2
- Match examples to the model’s training and the user’s actual input. In a coding copilot, plain-English few-shot instructions were a poor match for users supplying half-written code and for a code-focused model; example code was a proposed next test. section 10.4
- Consider finetuning alongside prompting when the task calls for it. Prompting is not always a substitute, and relevant examples cannot necessarily overcome a mismatch between the task and the model’s training. section 7.2.3 section 10.4
A useful instruction can cue a familiar pattern: asking for a step-by-step explanation can elicit a step-by-step response. Purposeful prompting can also ask a model to seek clarification when a request is unclear. Neither tactic makes the model inherently reliable; the prompt still needs to be tested against the intended interaction. section 2.1.2 section 7.2
Optimizing a math prompt
A DSPy example defines a math question as input and a numerical answer as output, then evaluates a zero-shot prompt against grade-school math problems. It uses chain-of-thought prompting and bootstrapped few-shot examples to optimize the prompt. The reported score rises from 14.5% to 74.5% in that evaluation without changing tokenization or finetuning the model. This is a measured result for that setup, not a promise that prompting eliminates hallucinations. section 12.2.5
config = dict(max_bootstrapped_demos=2) #1
class CoT(dspy.Module):
def __init__(self):
super().__init__()
self.prog = dspy.ChainOfThought(QASignature, max_tokens=1000)
def forward(self, question):
return self.prog(question=question)
print("Creating Bootstrapped Few Shot Prompt") #2
teleprompter = BootstrapFewShot(metric=gsm8k_metric, **config)
optimized_cot = teleprompter.compile(
CoT(), trainset=gsm8k_trainset, valset=gsm8k_devset
)
optimized_cot.save("optimized_llama3_math_cot.json")
print("Evaluating Optimized CoT Prompt") #3
evaluate(optimized_cot)
#149/200 74.5%
Inspect the DSPy chain-of-thought module, few-shot compilation, and evaluation call behind the reported optimized result. section 12.2.5
Keep prompt engineering distinct from prompt tuning: engineering refines the instructions used to guide responses, while prompt tuning is a training-time adaptation that can reuse a largely frozen model. For an application, the immediate rule is to test prompts with realistic users, inputs, and expected outputs before deciding whether prompt changes are enough. section 5.3.1 section 7.1 section 10.4
