liveBook · Manning

Artificial intelligence

As explained in Learn AI Data Engineering

Artificial intelligence is the field of creating computational systems that perform tasks associated with human intelligence, such as reasoning, learning, perception, and decision-making.

Artificial intelligence (AI) is used as a practical capability for automating, interpreting, and enriching work that would otherwise require repetitive human effort. Modern AI is presented as an enhancement to skilled human judgment: it can remove drudgery while leaving people responsible for creativity, critical thinking, problem-solving, and decisions about how to use its output. section 1.0

In data workflows, AI can act first as a coding companion—generating SQL or Python, reviewing prompts, debugging code, and comparing implementation options—and later as an embedded pipeline component. Programmatic use moves beyond interactive prompting toward ingestion, transformation, enrichment, and structured extraction at scale. section 1.1 section 1.2 section 5.0

AI is useful where inputs are variable, unstructured, or require contextual judgment, including:

  • Classifying and interpreting text, such as assigning sentiment labels to articles for downstream analysis. section 5.0
  • Cleaning and validating data by flagging missing values, invalid formats, contextual anomalies, duplicates, and reference mismatches through natural-language expectations. section 6.0
  • Extracting or reshaping difficult data, including nested JSON, messy text, entities, time-series fields, and account profiles. section 7.6
  • Replacing brittle site-specific scraping logic with schema-driven extraction from cleaned HTML, while assisting URL discovery and image selection. section 11.3

AI as a sentiment-enrichment step

Fetch articles, preprocess them into clean content blocks, send them to an AI API for structured sentiment labels, then store and log the results in a pandas DataFrame. The enriched data can continue to scikit-learn, PostgreSQL, or Tableau. Secure API keys with environment variables or a secrets manager as the workflow becomes operational. section 5.0

AI is not a default replacement for ordinary code. Prefer deterministic SQL, rules, or libraries for simple transformations, exact lookups, joins, and tasks whose scale makes model calls unjustified. For AI-produced data, plan validation through schemas, spot checks, known-value comparisons, confidence scoring, status tracking, review queues, and fallback logic. section 11.3 section 11.9

Figure 11.6 Flowchart: "Should I use AI for this step?" Decision tree walking through the key questions from the checklist.

Use the decision flowchart to evaluate whether a pipeline step has the unstructured input, variable logic, human-like judgment, economic value, and validation path that justify AI. section 11.9

How Learn AI Data Engineering covers it

How deep it goes
Explains applied selection, embedding, validation, costs, and fallbacks rather than model architectures or AI theory.
Examples
Uses sentiment-labelled articles, nested JSON, messy subscription codes, entity resolution, time transformations, and product-page extraction.
What it assumes
Assumes helpful familiarity with SQL, Python, and AI concepts while providing practical hands-on entry points.
What you can do after
Apply it via APIs and coding companions to enrich pipelines, extract structured data, and validate AI-generated results.
How it connects
Connects AI with data engineering, SQL, Python, data quality, large language models, web scraping, and ETL/ELT pipelines.

Also explained in

Compare how 29 books explain artificial intelligence →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Learn AI Data Engineering. The section links point to the parts of the book it draws on.