3 Working through an API: Generating text

 

This chapter covers

  • Generative AI models and their categorization based on specific applications
  • The process of listing available models, understanding their capabilities, and choosing the appropriate ones
  • The completion API and chat completion API offered by OpenAI
  • Advanced options for completion and chat completion APIs that help us steer the model and hence control the generation
  • The importance of managing tokens in a conversation for improved user experience and cost-effectiveness

We have seen that large language models (LLMs) provide a powerful suite of machine learning tools specifically designed to enhance natural language understanding and generation. OpenAI features two notable APIs: the completion and the chat completion APIs. These APIs, unique in their dynamic and effective text-generation capabilities, resemble human output. In addition, they offer developers exclusive opportunities to craft various applications, from chatbots to writing assistants. OpenAI was the first to introduce the pattern of completion and chat completion APIs, which now embody almost all implementations, especially when companies want to build generative-AI-powered tools and products.

3.1 Model categories

3.1.1 Dependencies

3.1.2 Listing models

3.2 Completion API

3.2.1 Expanding completions

3.2.2 Azure content safety filter

3.2.3 Multiple completions

3.2.4 Controlling randomness

3.2.5 Controlling randomness using top_p

3.3 Advanced completion API options

3.3.1 Streaming completions

3.3.2 Influencing token probabilities: logit_bias

3.3.3 Presence and frequency penalties

3.3.4 Log probabilities

3.4 Chat completion API

3.4.1 System role

3.4.2 Finish reason