liveBook · Manning

GPT-4o

As explained in Data Analysis with LLMs

GPT-4o is OpenAI's multimodal generative model in the GPT family.

GPT-4o is OpenAI’s multimodal model: it can work with text and images in the same request. A short natural-language instruction can ask it to write a story, summarize a document, generate code, or answer a question about a picture. For data analysis, this makes it useful when the task is easier to describe in words than to encode as a task-specific model. section 1.1 section 6.2

To call GPT-4o from Python, create an OpenAI client and send a chat completion request. Set `model` to `gpt-4o` and put the instruction in a message with the `user` role. The generated text is available in `result.choices[0].message.content`. Set up an OpenAI access key first; an environment variable is the recommended way to provide it. section 3.4 section 4.2.3

Listing 3.2 Using GPT-4o for chat completion
import openai          #1
client = openai.OpenAI()  #2

result = client.chat.completions.create(  #3
    model='GPT-4o',                    #4
    messages=[{                        #5
        'role':'user', 
        'content':'Tell me a story!'
        }])
print(result)

A minimal Python request shows where the model name and user instruction go. section 3.4

For an image question, place both the question and an image URL in one user message. Give each content element a type: `text` for the question and `image_url` for the picture. For example, a prompt can show an apple and ask whether the picture contains a banana; the expected answer is “No.” A message can also contain multiple images. section 6.2.1

{'role':'user', 'content':[
    {'type':'text', 'text':question},  #1
    {'type':'image_url', 'image_url':{
        'url':image_url}}          #2
    ]
}

The message structure makes the two input types explicit while keeping the question and image together. section 6.2.1

The same input-and-instruction pattern supports several practical workflows:

  • Compare a reference picture with a picture to tag, ask for “Yes” or “No,” and use the answer to label matching files. section 6.3.1 section 6.3.4
  • Select frames from a video and ask for a concise title. Each submitted frame adds to the cost, so frame selection matters. section 6.4.1
  • Adjust a story request with settings such as a token limit, stop sequence, temperature, and presence penalty to shape its length and style. section 3.5.4

Treat a model call as a remote operation, not a guaranteed step in a pipeline. A connection failure can interrupt a run over many inputs; retrying after a short delay makes such runs more resilient. When a task needs a fixed answer such as “Yes” or “No,” state that output requirement in the prompt and handle the returned answer in the surrounding code. section 4.2.3 section 6.3.4

How Data Analysis with LLMs covers it

How deep it goes
Walks through chat requests, multimodal message formats, customization, retries, and an end-to-end tagging workflow rather than model internals.
Examples
Uses a story request, an apple-versus-banana image question, paired-picture tagging, and titles generated from video frames.
What it assumes
Starts with client and message setup, while requiring an OpenAI access key for API calls.
What you can do after
Apply it via the OpenAI Python library to send chat and image questions, or build a picture-tagging workflow around model answers.
How it connects
Connects GPT-4o to chat completions, multimodal prompts, visual question answering, image tagging, and video titling.

Also explained in

Compare how 4 books explain GPT-4o →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Data Analysis with LLMs. The section links point to the parts of the book it draws on.