Part 2 Data analysis with language models

A web interface is sufficient to use language models on small pieces of data. But how can we scale it up to large data sets?

In chapter 3, we look at OpenAI’s Python library, a powerful library that enables you to call language models directly from Python. We discuss the most important library functions, including various parameters that can be used to fine-tune the behavior of the language model for your specific use case.

Chapter 4 discusses several mini-projects that use OpenAI’s language models via the Python library to analyze collections of text documents. For instance, our projects cover classifying text documents into one of several prespecified categories, extracting specific snippets of information from text, and clustering similar text documents together.

Chapter 5 shows how to use language models to analyze structured data: data tables and graph data. Here, language models are used as an interface, translating questions about the data (in natural language) into formal queries that can be processed by specialized analysis tools. For instance, the chapter demonstrates how to use language models to build interfaces that translate questions to queries in SQL that can be processed using relational database management systems.