liveBook · Manning

Schema

As explained in LLMs in Production

A schema is a formal specification of the structure, fields, types, and constraints of data.

Before preparing a pile of data for an LLM, decide what features it should contain and how they should be structured. That specification is its schema. It gives the later cleaning steps a consistent starting point. section 4.3.2

A practical preparation sequence is:

  • Determine a schema for the features. section 4.3.2
  • Normalize or scale features for the intended outcome, then check for bias and anomalies. section 4.3.2
  • Convert the data for model ingestion; for LLMs, this involves tokenization and embedding. section 4.3.2

For fine-tuning, the instruct schema is a common option: instruction tuning uses instructions to show a model how to perform a task. A schema is only the starting point, not a substitute for checking whether the data represents the problem. section 4.3.2

How LLMs in Production covers it

How deep it goes
Places schema selection in a preparation sequence but does not detail how to construct data pipelines.
Examples
Uses a pile of data needing a feature schema and identifies the instruct schema as a fine-tuning option.
What it assumes
Assumes familiarity with features and model training; does not teach data-pipeline construction.
What you can do after
Practice/perform selecting a feature schema before cleaning and converting data for model ingestion.
How it connects
Connects schemas to normalization, bias checks, tokenization, embedding, and instruction tuning.
Cover of LLMs in Production
liveBookmanning.com
Read LLMs in Production on liveBook Get the book at manning.com

Also explained in

Compare how 44 books explain schema →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLMs in Production. The section links point to the parts of the book it draws on.