Artificial intelligence (AI) is used as a practical capability for automating, interpreting, and enriching work that would otherwise require repetitive human effort. Modern AI is presented as an enhancement to skilled human judgment: it can remove drudgery while leaving people responsible for creativity, critical thinking, problem-solving, and decisions about how to use its output. section 1.0
In data workflows, AI can act first as a coding companion—generating SQL or Python, reviewing prompts, debugging code, and comparing implementation options—and later as an embedded pipeline component. Programmatic use moves beyond interactive prompting toward ingestion, transformation, enrichment, and structured extraction at scale. section 1.1 section 1.2 section 5.0
AI is useful where inputs are variable, unstructured, or require contextual judgment, including:
- Classifying and interpreting text, such as assigning sentiment labels to articles for downstream analysis. section 5.0
- Cleaning and validating data by flagging missing values, invalid formats, contextual anomalies, duplicates, and reference mismatches through natural-language expectations. section 6.0
- Extracting or reshaping difficult data, including nested JSON, messy text, entities, time-series fields, and account profiles. section 7.6
- Replacing brittle site-specific scraping logic with schema-driven extraction from cleaned HTML, while assisting URL discovery and image selection. section 11.3
AI as a sentiment-enrichment step
Fetch articles, preprocess them into clean content blocks, send them to an AI API for structured sentiment labels, then store and log the results in a pandas DataFrame. The enriched data can continue to scikit-learn, PostgreSQL, or Tableau. Secure API keys with environment variables or a secrets manager as the workflow becomes operational. section 5.0
AI is not a default replacement for ordinary code. Prefer deterministic SQL, rules, or libraries for simple transformations, exact lookups, joins, and tasks whose scale makes model calls unjustified. For AI-produced data, plan validation through schemas, spot checks, known-value comparisons, confidence scoring, status tracking, review queues, and fallback logic. section 11.3 section 11.9
Figure 11.6 Flowchart: "Should I use AI for this step?" Decision tree walking through the key questions from the checklist.
Use the decision flowchart to evaluate whether a pipeline step has the unstructured input, variable logic, human-like judgment, economic value, and validation path that justify AI. section 11.9




























