Chapter 4: Using a Coding Companion with Python
The chapter introduces using an AI coding companion to build and refine Python solutions for repetitive data engineering tasks while keeping the engineer in control.
After this chapter you will be able to
- Iterative Python prompting. Explain why Python workflows lend themselves to step-by-step collaboration with a coding companion and how clear examples and targeted prompts help refine the code.
- AI assistance with API work. Describe how an AI coding companion such as ChatGPT can streamline API interactions by generating boilerplate code, helping with edge cases, and summarizing API documentation.
- Zero-shot API code workflow. Explain how a zero-shot prompt asks the coding companion for Python code that queries a list of numbers, extracts facts from JSON, handles failed requests, and puts the results in a pandas DataFrame.
- Prompt clarity and code review. Explain why generated code should be treated as a draft and how specifying the execution environment and desired feedback can prevent overengineered logging or script-style code in a notebook.
- Iterative timeout retry refinement. Describe how a follow-up prompt in the same conversation refines the working API script to print each number as processing begins and retry timed-out requests up to three times with a five-second delay.
- Documentation grounded API prompting. Explain how uploading API documentation helps a coding companion generate code for documented parameters and endpoints, especially when an API is private or its documentation is too large to put in a prompt.
- NumbersAPI request requirements. Describe the documentation-based NumbersAPI prompt and its requested Python output, including JSON responses, the notfound=floor and fragment=true query parameters, a pandas DataFrame, and basic error handling. Describe the follow-up test that adds random alongside fixed numbers.
- Verification of generated API code. Explain how a companion can misread uploaded documentation and how to check its generated API requests and responses before relying on them.
- AI-assisted JSON flattening. Explain why nested API JSON can be tedious to unnest manually and how a well-prompted AI coding companion can generate Python code to flatten it into a pandas DataFrame.
- Single nested field extraction. Describe prompting a coding companion to fetch JSONPlaceholder users and create a pandas DataFrame containing name, email, and address.city. Explain that extracting address.city from each user record is a first step in flattening JSON and that .get() provides a safe access pattern.
- Deeply Nested Field Extraction. Describe how a prompt builds on a working JSON-flattening script to fetch JSONPlaceholder users and create a DataFrame containing fields from the user, company, and address objects.
- Unreliable JSON Structure Handling. Explain why code that works on one API response may fail on others, and describe safeguards for missing, null, empty, or differently named data.
- AI-assisted regex for phone numbers. Explain why regex is a good candidate for AI coding assistance and describe the planned phone-number example: prompting for Python regex to extract numbers, reformat them consistently, and refine the logic for extensions.
- Mixed-format phone number extraction. Describe how to ask an AI coding companion for a Python script that extracts U.S. phone numbers from text, and state that the regex-based result should find all three numbers despite their different punctuation.
- Phone number normalization. Describe how to prompt the coding companion, in the same chat, to modify the extraction script so it removes non-digit characters, normalizes only 10-digit numbers, and outputs them as +1-XXX-XXX-XXXX.
- Structured phone number extraction. Describe how a Python script can use regex capturing groups to extract U.S. phone numbers from text and split each match into area_code, first_3, and last_4 columns in a pandas DataFrame.
- Regex reliability and validation. Explain why AI-generated regex for phone numbers can fail on messy or unexpected input and how to check it before using it in a pipeline. Describe the practice exercise's international numbers as a case requiring changes to the U.S.-focused pattern and normalization logic.
What it builds on
- Few-Shot Prompting (chapter 3) · Introduces guiding a coding companion with examples.
- Zero-Shot Prompting (chapter 3) · Introduces zero-shot prompting.
Section by section
Introduction
The chapter introduces using an AI coding companion to build and refine Python solutions for repetitive data engineering tasks while keeping the engineer in control.
- Iterative Python prompting: Explain why Python workflows lend themselves to step-by-step collaboration with a coding companion and how clear examples and targeted prompts help refine the code.
Why it matters: This approach helps you use AI to speed up Python work without giving up control of the result. - Python task roadmap: Describe the kinds of data engineering work the chapter will address with Python and a coding companion: API interactions, nested JSON parsing, and complex regular expressions.
Why it matters: These examples show where AI assistance may save time on tedious, repetitive, error-prone work.
Key terms: coding companion, few-shot prompting, iterative prompting, targeted prompts, API interactions, nested JSON, regular expressions
4.1 Interacting with APIs Using AI Coding Companions & Python
Introduces why data engineers use APIs, what makes API work tedious, and how AI coding companions can help with Python API interactions.
- API data access and challenges: Explain that APIs provide structured data from online services for applications and automated workflows, while using them requires HTTP requests, JSON parsing, and error handling.
Why it matters: These are the tasks the Python API examples are meant to address. - AI assistance with API work: Describe how an AI coding companion such as ChatGPT can streamline API interactions by generating boilerplate code, helping with edge cases, and summarizing API documentation.
Why it matters: These are the specific ways the section proposes reducing tedious API work.
Key terms: APIs, HTTP requests, JSON responses, errors, AI coding companions, ChatGPT, boilerplate code, edge cases, API documentation
4.1.1 Fetching Data from an API
This section uses NumbersAPI to show how an AI coding companion can generate Python API code, then improve it through clearer prompts and iterative refinement.
- NumbersAPI request and response: Describe how changing the number in a NumbersAPI URL with `?json` retrieves a different fact while preserving a consistent JSON structure that can be parsed.
Why it matters: A predictable response structure makes the API a useful starting point for extracting data with Python. - Zero-shot API code workflow: Explain how a zero-shot prompt asks the coding companion for Python code that queries a list of numbers, extracts facts from JSON, handles failed requests, and puts the results in a pandas DataFrame.
Why it matters: It connects an API request to a usable analysis table while showing what to check in generated code. - Prompt clarity and code review: Explain why generated code should be treated as a draft and how specifying the execution environment and desired feedback can prevent overengineered logging or script-style code in a notebook.
Why it matters: Precise prompts and execution checks help you get code that fits the task rather than accepting a model’s assumptions. - Iterative timeout retry refinement: Describe how a follow-up prompt in the same conversation refines the working API script to print each number as processing begins and retry timed-out requests up to three times with a five-second delay.
Why it matters: Iterative prompting lets you add resilience and useful real-time feedback after confirming the basic API call works.
Key terms: NumbersAPI, JSON, query parameters, zero-shot prompt, requests, pandas DataFrame, error handling, overengineering, prompt ambiguity, Jupyter notebook
4.1.2 Enhancing API Calls with AI Coding Companions and API Documentation
This section shows how to use uploaded API documentation to prompt an AI coding companion for Python API code, then verify the code against the documentation and the live API.
- Documentation grounded API prompting: Explain how uploading API documentation helps a coding companion generate code for documented parameters and endpoints, especially when an API is private or its documentation is too large to put in a prompt.
Why it matters: Providing the actual reference reduces guesswork, particularly for APIs the companion cannot look up. - NumbersAPI request requirements: Describe the documentation-based NumbersAPI prompt and its requested Python output, including JSON responses, the notfound=floor and fragment=true query parameters, a pandas DataFrame, and basic error handling. Describe the follow-up test that adds random alongside fixed numbers.
Why it matters: Specific, documented requirements make the generated script more useful and give you concrete behavior to test. - Verification of generated API code: Explain how a companion can misread uploaded documentation and how to check its generated API requests and responses before relying on them.
Why it matters: Documentation reduces mistakes but does not replace checking what the API actually accepts and returns.
Key terms: API documentation, AI coding companion, upload, NumbersAPI, numbers_api_reference.txt, notfound=floor, fragment=true, JSON, pandas DataFrame, required parameters
4.2 Unnesting Complex JSON Objects with AI Companions & Python
Introduces AI-assisted flattening of nested API JSON into analysis-ready pandas DataFrames and the JSONPlaceholder dataset used to practice it.
- AI-assisted JSON flattening: Explain why nested API JSON can be tedious to unnest manually and how a well-prompted AI coding companion can generate Python code to flatten it into a pandas DataFrame.
Why it matters: Flattening nested records makes API data easier to analyze. - JSONPlaceholder practice data: Describe the JSONPlaceholder example and how the section proposes prompting ChatGPT about it.
Why it matters: The familiar sample API provides nested records to practice on without supplying a JSON sample in the prompt.
Key terms: nested JSON, unnesting, AI coding companions, pandas DataFrames, JSONPlaceholder API, address, company, 10 user records
4.2.1 Simple Example: Flattening a Single Nested Field
A simple JSONPlaceholder example shows how a coding companion can flatten one nested user field into a pandas DataFrame.
- Single nested field extraction: Describe prompting a coding companion to fetch JSONPlaceholder users and create a pandas DataFrame containing name, email, and address.city. Explain that extracting address.city from each user record is a first step in flattening JSON and that .get() provides a safe access pattern.
Why it matters: This small example establishes a pattern for working with more complex nested JSON.
Key terms: JSONPlaceholder, pandas DataFrame, address.city, flattening JSON, .get()
4.2.2 Complex Example: Extracting Deeply Nested & Combined Fields
This section shows how to use a coding companion to flatten deeply nested API data into a DataFrame while guarding against inconsistent JSON structures.
- Deeply Nested Field Extraction: Describe how a prompt builds on a working JSON-flattening script to fetch JSONPlaceholder users and create a DataFrame containing fields from the user, company, and address objects.
Why it matters: The same approach lets you turn layered API responses into columns without starting over for each new field. - Combined Geographic Coordinates: Explain how the latitude and longitude under address.geo are combined into one Geo string column, alongside separately extracted fields.
Why it matters: An output column can be constructed from multiple nested values, not just copied from one JSON key. - Unreliable JSON Structure Handling: Explain why code that works on one API response may fail on others, and describe safeguards for missing, null, empty, or differently named data.
Why it matters: API parsing needs to survive records whose shapes differ from the example used to generate the code.
Key terms: few-shot prompting, JSONPlaceholder, DataFrame, company.name, company.catchPhrase, address.city, address.geo, geo.lat, geo.lng, Geo
4.3 Using AI to Implement Regex Patterns
This section introduces AI-assisted regex as a way to extract and normalize messy text, using Python phone-number examples that will be refined for edge cases.
- AI-assisted regex for phone numbers: Explain why regex is a good candidate for AI coding assistance and describe the planned phone-number example: prompting for Python regex to extract numbers, reformat them consistently, and refine the logic for extensions.
Why it matters: The example shows how a plain-language request can become text-processing code that you can test and refine.
Key terms: regular expressions, regex, AI coding companions, Python’s re module, extensions
4.3.1 Extracting Phone Numbers from Text
The section shows how a conversational prompt can produce Python regex code that extracts U.S. phone numbers in different formats from a block of text.
- Mixed-format phone number extraction: Describe how to ask an AI coding companion for a Python script that extracts U.S. phone numbers from text, and state that the regex-based result should find all three numbers despite their different punctuation.
Why it matters: Extracting the numbers is the first step before normalizing or separating their components.
Key terms: Python script, U.S. phone numbers, regex
4.3.2 Normalizing Phone Numbers with Regex and AI
This section uses an AI coding companion to turn extracted phone numbers into a consistent format.
- Phone number normalization: Describe how to prompt the coding companion, in the same chat, to modify the extraction script so it removes non-digit characters, normalizes only 10-digit numbers, and outputs them as +1-XXX-XXX-XXXX.
Why it matters: Consistent phone numbers are easier to use in downstream data work.
Key terms: normalize, few-shot, non-digit characters, +1-XXX-XXX-XXXX
4.3.3 Extracting Number Components into a DataFrame
This section uses AI-generated regex to split U.S. phone numbers into structured fields, compares DataFrame and JSON output, and warns that regex must be tested against messy inputs.
- Structured phone number extraction: Describe how a Python script can use regex capturing groups to extract U.S. phone numbers from text and split each match into area_code, first_3, and last_4 columns in a pandas DataFrame.
Why it matters: Separating number components makes the extracted data easier to analyze and use downstream. - Regex reliability and validation: Explain why AI-generated regex for phone numbers can fail on messy or unexpected input and how to check it before using it in a pipeline. Describe the practice exercise's international numbers as a case requiring changes to the U.S.-focused pattern and normalization logic.
Why it matters: A plausible-looking regex can silently miss valid data or accept invalid matches. - DataFrame and JSON output choices: Distinguish between returning extracted phone components as a pandas DataFrame for tabular analysis and as a list of JSON objects for document-oriented workflows. State that a DataFrame can also be converted to JSON later.
Why it matters: Choosing an output shape that fits the destination can simplify downstream integration.
Key terms: capturing groups, area_code, first_3, last_4, pandas DataFrame, regex, test cases, real-world samples, phonenumbers, dateutil
4.4 Lab
This lab asks readers to use an AI coding companion and Jupyter Notebook to retrieve brewery data, clean phone numbers and website URLs with regex, and verify the results.
4.5 Lab Answers
The section gives three lab-answer prompts for fetching brewery data into a pandas DataFrame and using regex to format phone numbers and extract website domains.
Where this is used later
- (chapter intro) (chapter 5) · Earlier work used AI coding companions while developing Python code for API interactions.
- Handling Hierarchical and Nested Data Structures (chapter 7) · Applies extraction techniques to hierarchical and nested data structures.
- Complex Text Processing with Regular Expressions (chapter 7) · earlier use of AI to implement regex patterns
- Transforming News Data with AI (chapter 8) · Transforming extracted data for a pipeline requires attention to the structure of its output.
Concepts explained in this chapter
- data frameA pandas DataFrame is a two-dimensional labeled tabular data structure in the pandas library.
- NumbersAPINumbersAPI is a public web API that returns trivia, mathematical, date, and year-related facts about numbers.
- JSONJSON is a text-based data interchange format that represents structured values using objects, arrays, strings, numbers, booleans, and null.
- regular expressionA regular expression is a pattern language for matching, extracting, or replacing text according to specified character and sequence rules.
- requestsRequests is a Python library for sending HTTP requests and handling their responses.
- retryRetry logic is a control technique that repeats a failed operation according to defined attempt and delay policies.
- zero-shot promptingZero-shot prompting is a prompting technique that asks a model to perform a task without providing task-specific examples.
- phone numberA U.S. phone number is a telephone number represented according to numbering and formatting conventions used in the United States.
- PythonPython is a general-purpose programming language widely used for automation, data processing, and artificial-intelligence applications.
- edge caseAn edge case is an unusual, extreme, or boundary condition that can expose limitations or failures in a system or process.
- computational notebookJupyter is an open-source interactive computing project providing notebook-based tools for writing and executing code with explanatory content and outputs.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.
