liveBook · Manning

Context window

As explained in Build Python Web Apps with Streamlit

Context window length is the maximum number of tokens that a language model can process as input in a single prompt.

A context window is the maximum amount of information a language model can process in one prompt, measured in tokens. A token averages roughly three-quarters of a word, so a larger window can hold more supporting material. section 10.5.1

Why more context is not always better

Large windows can hold extensive knowledge bases, but sending everything with every question may overload the model, reduce performance, and increase token-based costs. Retrieval Augmented Generation instead retrieves only the information relevant to the user’s question before adding it to the prompt. section 10.5.1

How Build Python Web Apps with Streamlit covers it

How deep it goes
Defines token capacity and explains overload and cost trade-offs, but stops short of implementation limits or token-counting procedures.
Examples
Compares model capacities with large literary works and contrasts full knowledge-base prompts with retrieved context.
What it assumes
Recognizes prompts and tokens while learning why supplying all available information can be ineffective or expensive.
What you can do after
Apply Retrieval Augmented Generation to provide relevant knowledge without sending an entire knowledge base in every prompt.
How it connects
Connects context windows with prompts, tokens, language models, knowledge bases, and Retrieval Augmented Generation.
Cover of Build Python Web Apps with Streamlit
liveBookmanning.com
Read Build Python Web Apps with Streamlit on liveBook Get the book at manning.com

Also explained in

Compare how 22 books explain context window →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Build Python Web Apps with Streamlit. The section links point to the parts of the book it draws on.