Guidance is an open-source Microsoft library for replacing “prompt and pray” with programmatic control over language-model output. It constrains response spaces, defines stopping tokens, and supports complex prompt templates. section 7.3.2
Its core building blocks combine a wrapped model with generation functions such as gen and select. Reusable grammars, defined with the @guidance decorator, express composable language rules that generated text must follow. section 7.3.2
Guidance is useful when unconstrained generation creates operational risk:
- Set custom stopping tokens or regular-expression constraints for bounded outputs. section 7.3.2
- Select from allowed values when a response must match a finite vocabulary. section 7.3.2
- Attach tools that pause generation, execute a function, and resume generation with the result. section 7.4.1
- Guard tool inputs with grammars, while still handling hallucinated tools, malformed arguments, and runtime errors. section 7.4.1
Constrained generation with Falcon
A wrapped Falcon model can generate a short continuation, stop at a newline or punctuation mark, select a color from ["cyan", "grey", "purple"], or produce text matching an email regular expression. The same approach supports a parts-of-speech grammar that selects labels such as Noun, Verb, Adjective, or Adverb for words in a sentence.
