Syntax

As explained in How Large Language Models Work

Syntax is the set of structural rules that determines whether expressions or programs are valid in a language.

Syntax is the set of structural rules that makes an expression or program valid. A nearly correct answer still fails when brackets, special characters, or language rules are wrong. section 5.1.3

Generated text can violate syntax in several practical ways:

• A JSON response fails when opening and closing brackets do not match or special characters are encoded incorrectly. section 5.1.3
• Code fails to compile when it borrows valid-looking syntax from another language, such as Python syntax in Modula-3. section 5.1.3
• Programming languages can share patterns such as indentation, variables, function names, return statements, and operators without sharing identical syntax. section 6.1

Figure 6.2 Two similar samples of code written in the programming languages Python (left) and Java (right). These show how byte-pair encoding can identify similar tokens across different languages. The boxes show individual tokens. Standard tokenization methods for human languages do a reasonable job on code since it has many similarities to natural language.

Compare Python and Java code samples to see shared tokens and coding patterns alongside language-specific syntax. section 6.1

How How Large Language Models Work covers it

How deep it goes

Explains strict validity through JSON brackets, escaping, compilation failures, and cross-language differences.

Examples

Uses JSON formatting, Modula-3 and Python code, and Python-Java comparisons to demonstrate valid and invalid syntax.

What you can do after

Diagnose syntax failures in generated JSON and code by checking formatting rules and language-specific validity.

How it connects

Connects syntax with tokenization, programming languages, variable names, code comments, and embeddings.

liveBookmanning.com

Read How Large Language Models Work on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  LLMs in ProductionSyntax is the system of rules governing how words and other linguistic units are structured into phrases and sentences.
• 

  liveBookmanning.com
  Learn AI-Assisted Python Programming, Second EditionSyntax is the set of symbols, words, and rules governing the valid structure and notation of programs in a programming language.

Compare how 3 books explain syntax →

Related concepts

• natural languagepart of · Natural language is a human language that develops through social use and is used for communication, in spoken or written form.
• Pythonpart of · Python is a high-level, general-purpose, dynamically typed programming language known for readable syntax and automatic memory management.
• programming languagepart of · A programming language is a formal language with defined syntax and semantics for expressing computations and instructions that computers can execute or interpret.
• semanticscontrasts with · Semantics is the meaning and behavior conveyed or assigned to expressions, constructs, representations, or their combinations in a language or symbolic system.

This explainer was generated by Manning's knowledge graph from the text of How Large Language Models Work. The section links point to the parts of the book it draws on.