Appendix A. Setting Up Your Environment
Every example in this book runs on a small, standard toolkit: Python, Jupyter Lab, PostgreSQL, an OpenAI account, and a handful of free data APIs. This appendix walks you through installing and configuring each piece once, so you can follow along with any chapter without hunting for setup steps mid-lesson.
You do not need everything before Chapter 1. Set up the basics (Python, the repository, and Jupyter Lab) first, then add tools as the chapters call for them. Use the checklist at the end of Section A.1 to see what each part of the book requires.
The companion GitHub repository mirrors everything in this appendix and includes copy-paste-ready setup files: https://github.com/dave-melillo/data_eng_ai
A.1 What you need and when
The book is organized in parts, and each part adds a little to your toolkit. You can install everything up front, but most readers prefer to set up tools as they go.
Table A.1 Tools by part of the book
|
Part / Chapters
|
What you need
|
|
Part 1 — Coding companions (Ch. 1–4)
|
Python, the repo, Jupyter Lab, an AI coding companion (ChatGPT, Cursor, Windsurf, or GitHub Copilot), PostgreSQL + pgAdmin, and the Pagila sample database
|
|
Part 2 — The OpenAI API (Ch. 5–9)
|
All of the above, plus an OpenAI API key, the OpenAI Python SDK, and free data API keys (NewsAPI)
|
|
Part 3 — Web scraping pipelines (Ch. 10–12)
|
All of the above, plus the scraping libraries (BeautifulSoup, requests) and a SerpAPI key for AI-assisted URL discovery
|