liveBook · Manning

Prompt

As explained in Reinforcement Learning from Human Feedback

A prompt is an input text or instruction provided to a language model to elicit a response.

A prompt structures the information and context supplied to a language model. Basic interactions may need little design; advanced, one-off tasks can depend on careful wording, formatting, and context selection. Prompting is therefore both an implementation choice and a practical craft: it can substantially change evaluation results, although data and training improvements usually offer larger gains. section 16.1

Prompts serve several roles in model pipelines:

  • Represent an RLHF starting state: a dataset-sampled prompt is paired with a generated completion to form one rollout, usually without state transitions. section 3.1.3
  • Encode conversational roles through a chat template, including system instructions, user messages, assistant responses, separators, and a generation cue. section 4.1
  • Condition instruction-tuning data and downstream behavior; prompts work best when their distribution resembles the tasks the model will encounter. section 4.2
  • Declare available tools and their calling rules, allowing generation to alternate between model tokens, tool inputs, and injected outputs. section 13.2

Role-formatted conversation

A formatted interaction can begin with a system instruction such as “You are a friendly chatbot who always responds in the style of a pirate,” followed by a user question and an assistant-generation marker. The template converts these roles into special tokens before inference. section 4.1

<system>
You are a function-calling AI model. You are provided with function
 signatures within <functions></functions> XML tags. You may call one
 or more functions to assist with the user query. Don't make
 assumptions about what values to plug into functions.
</system>

<functions>
[
  {
    "name": "search_movies",
    "description": "Search for movies by title and return matching
     results with IDs.",
    "parameters": {
      "type": "object",
      "properties": {
        "query": {
          "type": "string",
          "description": "The search string for the movie title."
        }
      },
      "required": ["query"]
    }
  },
  {
    "name": "get_movie_details",
    "description": "Fetch detailed information about a movie including
     cast, runtime, and synopsis.",
    "parameters": {
      "type": "object",
      "properties": {
        "movie_id": {
          "type": "string",
          "description": "The unique identifier for the movie."
        }
      },
      "required": ["movie_id"]
    }
  },
  {
    "name": "get_showtimes",
    "description": "Get movie showtimes for a given location and date.",
    "parameters": {
      "type": "object",
      "properties": {
        "movie_id": {
          "type": "string",
          "description": "The unique identifier for the movie."
        },
        "zip_code": {
          "type": "string",
          "description": "ZIP code for theater location."
        },
        "date": {
          "type": "string",
          "description": "Date for showtimes in YYYY-MM-DD format."
        }
      },
      "required": ["movie_id", "zip_code"]
    }
  }
]
</functions>

<user>
...
</user>

Inspect a tool-use prompt that defines available functions, argument schemas, and instructions for producing reliable calls. section 13.2

How Reinforcement Learning from Human Feedback covers it

How deep it goes
Explains prompt effects, role formatting, RLHF structure, tool calls, and evaluation examples without deriving prompting algorithms.
Examples
Uses pirate-style system instructions, helicopter questions, multiple-choice reasoning prompts, tool schemas, and Fibonacci code execution.
What it assumes
Assumes familiarity with language-model generation and introduces role-based formatting, prompt-completion rollouts, and tool-conditioned interaction.
What you can do after
Apply it via chat templates, RLHF datasets, evaluation prompts, and tool-orchestration loops.
How it connects
Connects prompts to language models, RLHF, instruction tuning, chat templates, tool use, and evaluation.
Cover of Reinforcement Learning from Human Feedback
liveBookmanning.com
Read Reinforcement Learning from Human Feedback on liveBook Get the book at manning.com

Also explained in

Compare how 50 books explain prompt →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Reinforcement Learning from Human Feedback. The section links point to the parts of the book it draws on.