LogExtraction is a Pydantic model that gives AI-extracted log data a consistent structure: log type, date, and time. Each field is optional and represented as a string, making the schema suitable for varied log entries. section 7.1
Extending the schema with a message
For `ERROR 2025-06-09 12:34:56 Server failed to respond`, add an optional `message` field, update the extraction prompt, and verify that the output contains `log_type`, `date`, `time`, and `message`. section 7.1
A structured model helps expose common AI-extraction failure modes:
- Invented fields can appear even when the log contains no corresponding value. section 7.1
- Inconsistent formatting can make downstream automation unreliable unless the prompt requires explicit structured output. section 7.1
Listing 7.2 AI - Complex Text Processing with Regular Expressions
import openai #A
import os #A
from dotenv import load_dotenv #A
from pydantic import BaseModel #A
from typing import Optional #A
# Load API key from .env file #B
load_dotenv() #B
openai.api_key = os.getenv("OPENAI_API_KEY") #B
# Define the data model for extracted output #C
class LogExtraction(BaseModel): #C
log_type: Optional[str] #C
date: Optional[str] #C
time: Optional[str] #C
# Example log entries #D
logs = [ #D
"ERROR 2025-06-09 12:34:56 Server failed to respond", #D
"INFO 2025-06-09 12:35:56 User logged in", #D
"WARNING 2025-06-09 12:36:56 Disk space low" #D
]
# Prompt for AI to extract log_type, date, and time #E
row_prompts = [ #E
"You are a data extraction assistant. Extract the following from the log entry:\n"
"- log_type: The type of log message (e.g., ERROR, INFO, WARNING)\n"
"- date: Extract the date in YYYY-MM-DD format\n"
"- time: Extract the time in HH:MM:SS format\n"
"Return the result as a JSON object matching the LogExtraction structure."
for log in logs #E
]
# Process each log entry #F
for log, prompt in zip(logs, row_prompts): #F
try: #F
# Make the API call #F
completion = openai.beta.chat.completions.parse( #F
model="gpt-4o", #F
messages=[ #F
{"role": "system", "content": prompt}, #F
{"role": "user", "content": log} #F
], #F
response_format=LogExtraction #F
)
extracted = completion.choices[0].message.parsed.dict() #F
print(extracted) #F
except Exception as e: #F
print(f"Error processing log entry: {e}") #F
Inspect the Python implementation, including the Pydantic schema, sample logs, and extraction prompt. section 7.1
