A Python dictionary is a mutable map: it associates keys with values. That makes it useful when a program needs to find a value by a meaningful name, such as a message’s 'role' or an image’s filename. section 3.4 section 6.3.4
Describe a chat message
The dictionary {'role': 'user', 'content': 'Tell me a story!'} pairs each attribute name with its value. Put it in a list to form a chat history; the role identifies who wrote the message, and the content holds what they said. section 3.4
Dictionaries can also hold values more complex than a short string:
- For a question about an image, a message’s 'content' can be a list of dictionaries. Each element identifies its type as 'text' or 'image_url' and supplies the corresponding content. section 6.2.1
- When loading PNG files for image tagging, a dictionary maps each filename to its encoded image. Separate maps hold images to tag and images of people to look for. section 6.3.4
- For fine-tuning data, dictionaries describe user and assistant messages. A sample puts those messages in a 'messages' list, ready to be written in JSON-lines format. section 9.8
Listing 6.2 Tagging people in pictures stored locally
import argparse
import base64
import os
import requests
import shutil
def load_images(in_dir): #1
""" Loads images from a directory.
Args:
in_dir: path of input directory.
Returns:
directory mapping file names to PNG images.
"""
name_to_image = {}
file_names = os.listdir(in_dir)
for file_name in file_names:
if file_name.endswith('.png'):
image_path = os.path.join(in_dir, file_name)
with open(image_path, 'rb') as image_file:
encoded = base64.b64encode(image_file.read())
image = encoded.decode('utf-8')
name_to_image[file_name] = image
return name_to_image
def create_prompt( #2
person_image, image_to_label):
""" Create prompt to compare images.
Args:
person_image: image showing a person.
image_to_label: image to assign to a label.
Returns:
prompt to verify if the same person appears in both images.
"""
task = {'type':'text',
'text':'Do the images show the same person ("Yes"/"No")?'}
prompt = [task]
for image in [person_image, image_to_label]:
image_url = {'url':f'data:image/png;base64,{image}'}
image_msg = {'type':'image_url', 'image_url':image_url}
prompt += [image_msg]
return prompt
def call_llm(ai_key, prompt): #3
""" Call language model to process prompt with local images.
Args:
ai_key: key to access OpenAI.
prompt: a prompt merging text and local images.
Returns:
answer by the language model.
"""
headers = {
'Content-Type': 'application/json',
'Authorization': f'Bearer {ai_key}'
}
payload = {
'model': 'gpt-4o',
'messages': [
{'role': 'user', 'content': prompt}
],
'max_tokens':1
}
response = requests.post(
'https://api.openai.com/v1/chat/completions',
headers=headers, json=payload)
return response.json()['choices'][0]['message']['content']
if __name__ == '__main__': #4
parser = argparse.ArgumentParser() #5
parser.add_argument('peopledir', type=str, help='Images of people')
parser.add_argument('picsdir', type=str, help='Images to tag')
parser.add_argument('outdir', type=str, help='Output directory')
args = parser.parse_args()
people_images = load_images(args.peopledir)
unlabeled_images = load_images(args.picsdir)
for person_name, person_image in people_images.items(): #6
for un_name, un_image in unlabeled_images.items(): #7
prompt = create_prompt(person_image, un_image)
ai_key = os.getenv('OPENAI_API_KEY')
response = call_llm(ai_key, prompt)
description = f'{un_name} versus {person_name}?'
print(f'{description} -> {response}')
if response == 'Yes': #8
labeled_name = f'{person_name[:-4]}{un_name}'
source_path = os.path.join(args.picsdir, un_name)
target_path = os.path.join(args.outdir, labeled_name)
shutil.copy(source_path, target_path)
The image-loading code starts with an empty dictionary, adds an entry for each PNG filename, and returns the completed map. section 6.3.4
