liveBook · Manning

Structured data

As explained in LLM Evaluation and Alignment, The Foundational Ideas

A structured database is an organized store of data whose records and fields follow a defined schema.

Structured data stores records in a defined organization, such as a structured database. When it supplies context to an AI system, evaluation focuses on whether retrieval returns the right information rather than on unstructured-text relevance alone. section 6.4.2

Evaluate structured-database retrieval by checking:

  • Result completeness: whether the retrieval includes all records required for the task. section 6.4.2
  • Execution accuracy (EX): whether the SQL query returns the correct result. section 6.4.2
  • Task success rate (SR): whether the overall result is correct. section 6.4.2

Evaluating a database-backed answer

For a question answered through SQL, verify that the query executes, returns the correct records, includes the required records, and produces a correct final result. Faithfulness and answer relevance can still be assessed after the database output enters the model context, but they do not replace database-specific checks. section 6.4.2

How LLM Evaluation and Alignment, The Foundational Ideas covers it

How deep it goes
Distinguishes completeness, execution accuracy, and task success, but does not detail database schema or query implementation.
Examples
Uses SQL queries that return correct records as the central evaluation artifact.
What you can do after
Diagnose database-backed retrieval by measuring completeness, query execution accuracy, and overall task success.
How it connects
Connects structured databases with SQL queries, retrieval-augmented generation, execution accuracy, and task success rate.

Also explained in

Compare how 31 books explain structured data →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of LLM Evaluation and Alignment, The Foundational Ideas. The section links point to the parts of the book it draws on.