How deeply does LLMs in Production cover instruct schema?

Defines three fields and works through a translation example, without specifying a dataset serialization or training procedure.

Instruct schema

As explained in LLMs in Production

An instruct schema is a structured data format that separates an instruction, its relevant input or metadata, and the desired response.

A task and its answer do not always tell a model what it was supposed to do. An instruct schema makes the task explicit by separating the instruction, relevant input, and desired response. It is a common way to prepare data for instruction tuning, especially when finetuning an LLM. section 4.3.2

Each example has three parts:

Instruction: the task given to the model. section 4.3.2
Input: relevant material or metadata for that task. section 4.3.2
Response: the output the model should produce. section 4.3.2

Translation example

Instruction: “Translate this sentence to Japanese.” Input: the sentence to translate. Response: its Japanese translation. Keeping the fields separate lets the example show both what the model was asked to do and what a suitable answer looks like. section 4.3.2

This structure can prepare a model for prompting and prompt tuning, but creating demonstrations takes more work than collecting ordinary crawl data. The pattern is not limited to text: an image–instruction–answer example can also serve as instruction-tuning data. Check that the examples represent the tasks and kinds of responses the model needs to handle. section 4.3.2 section 4.6

How LLMs in Production covers it

liveBook manning.com

Read LLMs in Production on liveBook Get the book at manning.com

Related concepts

fine-tuninguses · Fine-tuning is adapting a pretrained machine-learning model by training it further on targeted data or objectives for a task, domain, or behavior.

This explainer was generated by Manning's knowledge graph from the text of LLMs in Production. The section links point to the parts of the book it draws on.