liveBook · Manning

Pipeline

Also called pipe

A pipeline is an ordered sequence of connected processing stages through which data or work flows, with each stage passing or transforming its output for subsequent stages.

How 63 Manning books explain it

Natural Language Processing in Action, Second Edition

An NLU pipeline is a sequence of natural-language-understanding stages that transform text into an interpretation or decision.

How it is used
Organizes text processing into stages for intent recognition, conversational responses, and question answering.
How deep it goes
Explains chatbot subsystems and implements simple and neural pipelines, including tokenization choices and CNN tensor shapes.
Examples
Uses a keyword greeting recognizer, spaCy token inspection, and a tutoring chatbot to illustrate pipeline behavior.
What you can do after
Builds pipelines from simple Python rules and applies spaCy and LangChain to tokenization and conversational tasks.

Read how Natural Language Processing in Action, Second Edition explains pipeline →

Build AI-Enhanced Web Apps

A workflow is an ordered set of connected tasks or operations that accomplishes a broader process.

How it is used
Orchestrates preprocessing, model calls, retrieval, and output handling in AI web applications.
How deep it goes
Explains runnable composition, sequential and parallel execution, streaming, and input matching with implementation examples.
Examples
Demonstrates vowel counting, document retrieval, streaming UI updates, and MapReduce document summarization.
What you can do after
Apply pipelines via LangChain.js to compose processing steps and integrate streamed results into a web interface.

Read how Build AI-Enhanced Web Apps explains pipeline →

AI Agents and Applications

A multi-step workflow is a process composed of several ordered or coordinated operations that collectively accomplish a task.

How it is used
Coordinates LLM application steps for summarization, web research, and retrieval-augmented question answering.
How deep it goes
Builds pipelines from individual components through parallel sub-chains, then examines linear-flow and state-management limitations.
Examples
Uses a web research report and a document-based question-answering chatbot to demonstrate composed operations.
What you can do after
Apply it via LangChain and LCEL to compose, invoke, and parallelize LLM workflows.

Read how AI Agents and Applications explains pipeline →

Crack Any Codebase with AI

A workflow is a repeatable sequence of predefined steps that systematically transforms inputs into consistent outputs.

How it is used
Automates systematic codebase exploration when manually asking repeated questions no longer scales.
How deep it goes
Explains fixed pipelines, repository triage, sequential context, and overview-then-zoom-in analysis, but stops short of autonomous decisions.
Examples
Demonstrates per-file Python summaries and a VS Code tutorial generator with abstraction maps, chapters, diagrams, and cross-references.
What you can do after
Build it from scratch as a script that scans a repository and generates consistent, structured codebase documentation.

Read how Crack Any Codebase with AI explains pipeline →

Machine Learning System Design

An ordered set of computational steps and dependencies that accomplishes a defined process.

How it is used
Organizes model preparation and prediction into connected, reproducible workflows rather than treating model training as the whole system.
How deep it goes
Defines pipeline DAGs and training–inference boundaries, then develops orchestration, scaling, configurability, and testing practices without detailing a particular platform.
Examples
Uses retail demand forecasting and multimodal image search designs, alongside a training-pipeline smoke test.
What you can do after
Evaluate and design training and inference workflows, including their tooling, scaling, reproducibility, and testing tradeoffs.

Read how Machine Learning System Design explains pipeline →

Delivering Machine Learning Projects: From design to deployment

A production data flow is the operational movement and transformation of data through a deployed machine learning system.

How it is used
Connects data preparation for exploration and modeling with production data flows for inference.
How deep it goes
Defines workflow-managed DAGs and outlines lifecycle tasks, instrumentation, and production integration without a complete implementation.
Examples
Illustrates an ELT workflow that joins and cleans two tables, and a shopping basket flow combining click, basket, and user data.
What you can do after
Evaluate and understand how to plan, instrument, and carry data pipelines from development into production.

Read how Delivering Machine Learning Projects: From design to deployment explains pipeline →

"Looks Good to Me"

A workflow is an ordered sequence of activities and handoffs used to complete a process.

How it is used
Organizes code review into shared stages, responsibilities, handoffs, feedback, and approval decisions.
How deep it goes
Defines four core workflow components and six team decisions, while deferring deeper automation and advanced practices.
Examples
Uses pull requests, reviewer assignments, review comments, approval gates, whiteboard maps, and revised baseline workflows.
What you can do after
Build it from scratch by agreeing on goals, tools, stages, guidelines, signoff conditions, and later refinements with the team.

Read how "Looks Good to Me" explains pipeline →

Financial AI in Practice

An ordered sequence of data-processing or modeling steps that transforms inputs into outputs.

How it is used
Uses pipelines to prepare financial data and connect models to credit, fraud, retrieval, and personalization workflows.
How deep it goes
Explains ordered stages and operational controls, with a minimal Airflow DAG that stops short of a complete production pipeline.
Examples
Uses a monthly credit data mart, a streaming fraud feature flow, and a news-to-JSON RAG workflow.
What you can do after
Apply it via Apache Airflow to organize a simplified monthly credit-data workflow, while identifying controls needed for production.

Read how Financial AI in Practice explains pipeline →

Local Image Generation

A workflow is an organized sequence or graph of processing components that produces an output through configured operations.

How it is used
Organizes Stable Diffusion generation for configurable image creation, repeated runs, and more complex multi-step tasks.
How deep it goes
Defines pipelines and builds ComfyUI graphs from fundamental nodes, while deferring many nodes and configuration details.
Examples
Demonstrates a Diffusers image call, img2img ramen upscaling, Flux workflow loading, and SDXL checkpoint-merging previews.
What you can do after
Operate/configure it in Diffusers or ComfyUI to run, reuse, share, and extend image-generation processes.

Read how Local Image Generation explains pipeline →

Effective Platform Engineering

A pipeline workflow is a defined arrangement of automation steps or jobs that executes as a coordinated process.

How it is used
Coordinates repository automation and platform releases, while also describing type-specific observability data ingestion.
How deep it goes
Explains triggers, filters, job dependencies, and reusable code; extends to platform-scale patterns without specifying a complete pipeline configuration.
Examples
Uses CircleCI branch and tag filters, control-plane service and extension pipelines, and preview-environment conformance tests.
What you can do after
Build it from scratch with simple triggers and orchestration, then move reusable steps into versioned shared code.

Read how Effective Platform Engineering explains pipeline →

Machine Learning Platform Engineering

An orchestrated pipeline is a coordinated sequence of automated data and machine learning workflow steps, including iteration and dependencies.

How it is used
Coordinates repeatable ML work from data preparation through validation and deployment, replacing error-prone manual handoffs with orchestrated execution.
How deep it goes
Builds from reusable components to compiled KFP workflows, then identifies production triggers, monitoring, and data-access concerns.
Examples
Uses dataset download and splitting, an ID card detector, and a configurable movie-recommender training pipeline.
What you can do after
Build it from scratch with KFP components, then connect and configure them as an executable ML workflow.

Read how Machine Learning Platform Engineering explains pipeline →

ASP.NET Core in Action, 3e

A handler pipeline is an ordered chain of HTTP message handlers that can modify requests and inspect responses around a final transport handler.

How it is used
Centralizes cross-cutting behavior for outgoing HTTP requests and responses.
How deep it goes
Explains request flow, handler lifetime, registration order, and custom handler setup, with a retry policy example.
Examples
Uses an API-key header handler and a Polly retry policy on a typed exchange-rates client.
What you can do after
Apply it via IHttpClientFactory to register custom handlers and a transient-error retry policy.

Read how ASP.NET Core in Action, 3e explains pipeline →

Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI

A data pipeline is a sequence of data-processing steps that moves and transforms data between systems or stages.

How it is used
Organizes tabular preprocessing and repeated model training or deployment into consistent workflows.
How deep it goes
Distinguishes three pipeline meanings and develops feature transformations through a Vertex AI deployment workflow.
Examples
Uses synthetic classification, Airbnb listings, and Kuala Lumpur real-estate prediction workflows.
What you can do after
Apply it via Scikit-learn or Vertex AI to transform data, evaluate models, and automate training and deployment.

Read how Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI explains pipeline →

Learn Docker in a Month of Lunches, Second Edition

A workflow is a declarative automation definition that specifies jobs, triggers, and execution steps for a software delivery process.

How it is used
Automates build, test, image promotion, and deployment checks after source changes.
How deep it goes
Shows a runnable job and a staged delivery process, while keeping most execution logic in scripts rather than workflow YAML.
Examples
Uses a PowerShell build job, staging-image workflows, and a temporary Kubernetes deployment test.
What you can do after
Build it from scratch with GitHub Actions YAML and reusable scripts, then verify runs and failures after pushing changes.

Read how Learn Docker in a Month of Lunches, Second Edition explains pipeline →

AI Agents in Action, Second Edition

Ingestion is the phase of preparing source documents by loading, transforming, embedding, and storing them for later retrieval.

How it is used
Prepares documents for later retrieval in retrieval-augmented generation.
How deep it goes
Outlines loading, chunking, embedding, and storage without an implementation walkthrough.
Examples
Uses a PDF and a capital-of-France query to illustrate document ingestion and retrieval.
What you can do after
Evaluate/understand how an ingestion pipeline prepares documents for retrieval.

Read how AI Agents in Action, Second Edition explains pipeline →

The Art of Code

A pipeline is a composable sequence of data-processing stages in which intermediate operations define transformations and a terminal operation executes them.

How it is used
Uses pipelines to express data transformations as a readable, declarative flow.
How deep it goes
Explains lazy stages, terminal execution, exception handling, and readability refactoring, while deferring deeper graceful-failure treatment.
Examples
Uses recent-book author extraction, file reading, user-email processing, and a people-list exercise.
What you can do after
Apply it via Java streams to filter and map data, execute a terminal operation, and simplify a long chain.

Read how The Art of Code explains pipeline →

Designing Deep Learning Systems: A guide for software engineers

A data pipeline is an automated workflow that repeatedly acquires, transforms, labels, or otherwise moves data through processing stages.

How it is used
Automates continuous ingestion, processing, labeling, and dataset updates for production machine learning workflows.
How deep it goes
Explains stages, automation, versioning, provenance, and a containerized Pachyderm pipeline definition.
Examples
Demonstrates OpenCV edge extraction from an images dataset and training triggered by newly labeled images.
What you can do after
Operate/configure it with Pachyderm to chain containers, trigger jobs from data changes, and inspect versioned outputs.

Read how Designing Deep Learning Systems: A guide for software engineers explains pipeline →

Spring AI in Action

A workflow is an explicitly defined sequence of operations that coordinates application processing steps.

How it is used
Coordinates Embabel actions into goal-oriented application processing.
How deep it goes
Derives workflows from typed action dependencies and goal annotations, stopping short of describing the framework’s internal planning algorithm.
Examples
Uses board-game rules, mechanics, player counts, Java records, TikaDocumentReader, and prompt templates.
What you can do after
Build it from scratch by composing typed Embabel actions that load inputs and achieve analysis goals.

Read how Spring AI in Action explains pipeline →

Go by Example

How it is used
Structures concurrent request processing behind a synchronous SendN API while delivering results through an iterator.
How deep it goes
Works through designing and integrating producer, optional throttler, and dispatcher stages, while deferring basic goroutine and channel knowledge.
Examples
Uses an HTTP load-testing client whose producer, throttler, and dispatcher send requests and stream Result values.
What you can do after
Build it from scratch by composing channel-connected stages that concurrently process requests without changing the exported API.

Read how Go by Example explains pipeline →

F# in Action: A Revised Edition of Get Programming with F#

A pipeline is a sequence of transformations in which the output of one step becomes the input of the next, applied here to HTTP requests and responses.

How it is used
Organizes data transformations, including HTTP request/response handling, as stages whose results feed subsequent stages.
How deep it goes
Explains forward-pipe syntax and argument flow, works through a collection query, and gives a procedure for inspecting intermediate results.
Examples
Uses arithmetic chains, an away-win query, a customer workflow, and partial execution of a query in FSI.
What you can do after
Apply it via F#’s `|>` operator to chain compatible functions and diagnose collection pipelines by inspecting intermediate results in FSI.

Read how F# in Action: A Revised Edition of Get Programming with F# explains pipeline →

Shipping Go: Develop, deliver, discuss, design, and go again

A software delivery pipeline is an ordered set of automated stages that processes code from a change through validation and delivery.

How it is used
Moves small code changes through automated feedback toward delivery, helping teams ship useful increments.
How deep it goes
Traces initial setup through automated checks and a produced binary, while treating individual phases as adaptable rather than prescribing a universal configuration.
Examples
Uses a hello API, a README milestone checklist, a Go endpoint, and a GitHub Actions binary.
What you can do after
Build it from scratch: establish a basic repository-to-artifact path, then extend its checks and delivery steps as needs become clearer.

Read how Shipping Go: Develop, deliver, discuss, design, and go again explains pipeline →

Platform Engineering on Kubernetes

A workflow is an organized sequence of activities and practices for accomplishing a software delivery or operational objective.

How it is used
Automates repeatable service-delivery tasks and organizes event processing for platform metrics.
How deep it goes
Explains definitions, engine-run instances, and task status, then works through Tekton resources and commands.
Examples
Uses a Tekton file-fetching Pipeline with task-run output and a CloudEvents-based metrics collection pipeline.
What you can do after
Apply it via Tekton to define and run a task sequence, then inspect its task runs and logs.

Read how Platform Engineering on Kubernetes explains pipeline →

AI-Powered Search

A pipeline is an ordered sequence of processing stages in which each stage transforms or passes information to the next stage.

How it is used
Organizes indexing and query interpretation into modular stages that can be rearranged or replaced for experimentation.
How deep it goes
Details parsing, enrichment, transformation, and search, while deferring embedding-based semantic search and question answering.
Examples
Illustrates restaurant queries and a question-answering silver set built from question–context pairs.
What you can do after
Apply it via the Transformers library to generate question-answering guesses for human review.

Read how AI-Powered Search explains pipeline →

Learn Concurrent Programming with Go

A pipeline is a sequence of processing stages connected so that the output of one stage becomes the input of the next.

How it is used
Organizes dependent concurrent tasks into stages to increase throughput.
How deep it goes
Implements channel-connected goroutine stages and examines throughput, latency, and bottlenecks through timed pipeline runs.
Examples
Uses RFC URL downloading and word extraction, plus a timed cupcake-factory workflow.
What you can do after
Build it from scratch with Go goroutines and channels, then target the bottleneck to improve throughput.

Read how Learn Concurrent Programming with Go explains pipeline →

The Art of AI Product Development

A workflow is an ordered, often iterative sequence of tasks and operations that produces a process outcome.

How it is used
Connects data preparation, prompt chaining, and agent automation to outcomes in AI products.
How deep it goes
Describes linked subtasks and workflow responsibilities without specifying a complete implementation.
Examples
Uses a financial analytics chatbot and a comparison of manual and agent-managed workflows.
What you can do after
Evaluate and understand how earlier workflow decisions affect user-facing outcomes and where automation may reduce manual work.

Read how The Art of AI Product Development explains pipeline →

Build an AI Agent (From Scratch)

A workflow is a predefined sequence of computational steps that controls execution while delegating selected steps to language models or other components.

How it is used
Structures predictable LLM and multi-agent processes while reserving dynamic behavior for stages that need it.
How deep it goes
Explains workflow selection, sequencing, context handling, nesting, and iteration through composable agent interfaces.
Examples
Uses presentation generation, support routing, document processing, and a research-to-editing report pipeline.
What you can do after
Evaluate/understand when to choose a workflow and how to combine predictable stages with agent behavior.

Read how Build an AI Agent (From Scratch) explains pipeline →

Build AI Drug Discovery Pipelines

A sequential pipeline is an architecture in which distinct processing stages execute in order and pass each stage's outputs to the next stage.

How it is used
Uses sequential stages to narrow compound libraries and connect target discovery, molecular generation, and clinical-trial design.
How deep it goes
Explains stage ordering, computational trade-offs, and platform handoffs, but stops short of implementation details for docking and molecular dynamics.
Examples
Uses hierarchical virtual screening, antimalarial hit detection, and a target-to-molecule platform as concrete pipeline examples.
What you can do after
Apply it to organize compound prioritization from inexpensive filters through ligand-based screening, docking, and molecular dynamics.

Read how Build AI Drug Discovery Pipelines explains pipeline →

Data Pipelines with Apache Airflow, Second Edition

A data workflow is an ordered set of data-processing activities and their dependencies.

How it is used
Coordinates dependent data-processing tasks, particularly recurring or batch-oriented work.
How deep it goes
Works from task boundaries and DAG-file definitions through scheduling and execution components, while leaving complex service configuration to individual pipelines.
Examples
Uses a rocket-launch download workflow and a handwritten-digit classifier that copies S3 data, trains with SageMaker, and deploys a model.
What you can do after
Build it from scratch as an Airflow DAG with ordered tasks and a schedule, then monitor its runs.

Read how Data Pipelines with Apache Airflow, Second Edition explains pipeline →

Build a Frontend Web Framework (From Scratch)

An operation sequence is an ordered collection of array-editing operations that describes a transformation from one array state to another.

How it is used
Plans child-node additions, removals, and moves so virtual DOM reconciliation can patch the real DOM.
How deep it goes
Works through a left-to-right algorithm and incremental JavaScript implementation, including current and original indices; leaves applying sequences as an exercise.
Examples
Uses [A, B, C] becoming [C, B, D] and a duplicate-containing array exercise to test generated operations.
What you can do after
Build it from scratch with arraysDiffSequence(), then verify its operations by applying them to an old array.

Read how Build a Frontend Web Framework (From Scratch) explains pipeline →

Deep Learning with R, Third Edition

A workflow is an organized sequence of modeling or training operations that provides a particular balance of usability and flexibility.

How it is used
Organizes model building, training, and input preparation around choices between convenience and control.
How deep it goes
Explains image preprocessing steps and dataset transformations, then describes interleaving and prefetching for text input.
Examples
Uses directory-based image datasets and a sharded text pipeline for mini-GPT pretraining.
What you can do after
Apply it via Keras and tf.data to prepare image batches and text sequences for model training.

Read how Deep Learning with R, Third Edition explains pipeline →

Functional Programming in JavaScript

A Ramda function that composes functions from left to right into a new function.

How it is used
Connects functional transformations into reusable sequences instead of confining operations to one object’s methods.
How deep it goes
Explains inputs, outputs, type mappings, and coupling, but stops short of implementing a general pipeline utility.
Examples
Uses string transformations, an isEmpty function, and a Lodash array chain involving filter, map, uniq, and sort.
What you can do after
Apply it by arranging functions whose outputs match the next functions’ inputs, including functions from different libraries.

Read how Functional Programming in JavaScript explains pipeline →

Grokking AI Applications

A data pipeline is an ordered process that ingests, transforms, and delivers data between systems or storage destinations.

How it is used
Connects structured data systems with LLM reasoning and reformats inputs and outputs for downstream use.
How deep it goes
Describes data connections and transformation examples, including table joins, aggregation, and text or numeric manipulation.
Examples
Uses files, systems, and users as input sources, and illustrates splitting fields and joining or aggregating tables.
What you can do after
Operate KNIME workflows to connect sources, transform data, and save results for downstream use.

Read how Grokking AI Applications explains pipeline →

JavaScript in Depth

A sequence of connected stream operations through which data flows from a source toward a destination.

How it is used
Organizes streamed data flow from a producer through transforms to a consumer while coordinating pacing and resource handling.
How deep it goes
Explains pipeline roles, pacing, buffers, errors, conversions, and per-chunk costs, but defers general runtime execution pipelines.
Examples
Demonstrates chained Web Stream transforms, buffered Node.js transforms, and conversions between Node.js and Web Streams.
What you can do after
Build it from scratch by chaining producers, transforms, and consumers while preserving backpressure and handling model-specific errors.

Read how JavaScript in Depth explains pipeline →

Data Analysis with LLMs

A data-processing pipeline is an ordered sequence of computational stages that transform or analyze data, with each stage passing results to the next.

How it is used
Organizes multistep language-model applications by passing data through connected processing components.
How deep it goes
Works through a three-component classification chain and its CSV application, while deferring parallel and conditional execution.
Examples
Uses sentiment classification from a CSV file, speech translation, and indexed-data question answering.
What you can do after
Apply it via LangChain to classify a batch of input texts and save the labels alongside CSV data.

Read how Data Analysis with LLMs explains pipeline →

Build a DeepSeek Model (From Scratch)

A training pipeline is an organized sequence of data preparation, model computation, and optimization operations for training a machine-learning model.

How it is used
Organizes data preparation, model computation, optimization, generation, and scaling into a runnable foundation-model workflow.
How deep it goes
Builds the pipeline from dataset preparation through scripts and parallel execution, while deferring production-scale infrastructure details.
Examples
Uses TinyStories preparation, MiniDeepSeek training and sampling, DualPipe scaling, reinforcement-learning workflows, and PyTorch distillation loops.
What you can do after
Build it from scratch as a data-to-generation training system and extend it with optimization and distributed-execution techniques.

Read how Build a DeepSeek Model (From Scratch) explains pipeline →

The Coder Cafe

A processor pipeline is an ordered sequence of stages through which instruction processing progresses.

How it is used
Explains how CPUs increase instruction throughput and why code can lose performance when a pipeline stalls.
How deep it goes
Works through a simplified four-stage timing model while noting that real CPU microarchitectures can use different pipeline depths.
Examples
Compares two sequential instructions with overlapped stages and uses a 20-stage pipeline to illustrate refill cost.
What you can do after
Evaluate/understand why overlapping instructions improves throughput and why conditional branches can waste cycles.

Read how The Coder Cafe explains pipeline →

Build Applications with Local AI Models on a Mac

A sequence of processing stages in which each stage feeds its output to the next.

How it is used
Connects audio capture, transcription, and model inference into a local voice assistant.
How deep it goes
Explains stage handoffs, temporary-file cleanup, and individual diagnostic checks; points to branching chains as a next step.
Examples
Uses a voice chatbot, a prompt-to-model chain, and a fine-tuning-to-deployment workflow as examples.
What you can do after
Apply it via LangChain to connect a prompt, model, and output parser, or diagnose a chatbot by testing prerequisites separately.

Read how Build Applications with Local AI Models on a Mac explains pipeline →

Effective Data Analysis

A data pipeline is an automated sequence of processing steps that retrieves, transforms, and delivers data between systems.

How it is used
Supports repeatable retrieval, transformation, and warehouse delivery while helping stakeholders choose between automation and manual collection.
How deep it goes
Compares connectors, custom workflows, manual uploads, and scraping, but stops short of detailed implementation code or orchestration design.
Examples
Uses weather retrieval, weekly source refreshes, survey spreadsheets, static census data, and scraped HTML as pipeline cases.
What you can do after
Evaluate and recommend retrieval approaches, then select connector, custom workflow, manual upload, or scraping strategies for warehouse use.

Read how Effective Data Analysis explains pipeline →

Deep Learning with Python, Third Edition

A supported way of using Keras that combines model-building and training choices for a particular level of flexibility.

How it is used
Connects data loading, preprocessing, batching, model training, and evaluation while scaling from convenient APIs to custom workflows.
How deep it goes
Explains file decoding, tensor conversion, resizing, batching, iteration, mapping, parallelization, prefetching, and direct fit() integration.
Examples
Uses directory-based cat-and-dog image classification, NumPy arrays, batching, mapped reshaping, augmentation, metric curves, and test evaluation.
What you can do after
Apply it via Keras utilities and Dataset operations to prepare inputs, train models, inspect metrics, and evaluate saved checkpoints.

Read how Deep Learning with Python, Third Edition explains pipeline →

Kafka Streams in Action, Second Edition

Pipelining is a processing pattern that overlaps record retrieval and record processing so the consumer can continue retrieving batches while earlier batches are handled.

How it is used
Overlaps high-volume Kafka consumption with slow asynchronous record processing to help a consumer stay caught up.
How deep it goes
Explains the consumer loop, partition offset calculation, and commit timing, while omitting the complete processor implementation.
Examples
Uses Java consumer and processor listings that poll batches, process partition records, queue offsets, and commit synchronously.
What you can do after
Implement pipelined Kafka consumption with manual commits that acknowledge only successfully processed records.

Read how Kafka Streams in Action, Second Edition explains pipeline →

Domain-Specific Small Language Models

A workflow in which an agent chains outputs from one step into later steps to complete a task.

How it is used
Connects agent tool outputs across steps to complete complex tasks.
How deep it goes
Defines output chaining and relates it to planning and tool use, without detailing a pipeline implementation.
Examples
Contrasts agent output chaining with a Transformers text-generation pipeline used for CodeGen inference benchmarks.
What you can do after
Evaluate/understand it as a way to carry tool results between agent steps.

Read how Domain-Specific Small Language Models explains pipeline →

Practical Automation with PowerShell: Effective scripting from the console to the cloud

A PowerShell pipeline passes output objects from one command as input to subsequent commands or functions.

How it is used
Chains operational cmdlets for archival, bulk server updates, and disk provisioning.
How deep it goes
Explains pipeline tradeoffs and function lifecycle blocks, while deferring broader pipeline internals.
Examples
Demonstrates file archiving, VMware cluster server updates, and raw-disk initialization.
What you can do after
Apply it via PowerShell cmdlets and functions to process collections without repeated queries.

Read how Practical Automation with PowerShell: Effective scripting from the console to the cloud explains pipeline →

C++ Concurrency in Action, Second Edition

A pipeline is a processing pattern in which work passes through a sequence of stages.

How it is used
Processes repeated sequences of operations across independent items by assigning work to stages.
How deep it goes
Explains stage threads and queue handoffs, then motivates the performance tradeoff without detailing synchronization implementation.
Examples
Uses network arrivals, filesystem scanning, and a 20-item workload to illustrate pipeline use and performance comparison.
What you can do after
Apply a pipeline by assigning operations to threads and passing completed items through queues.

Read how C++ Concurrency in Action, Second Edition explains pipeline →

Introduction to Generative AI, Second Edition

A data pipeline is a sequence of processes that collects, transforms, transports, and prepares data for downstream use.

How it is used
Uses linked prompts to divide a larger task into steps, especially in enterprise applications, RAG workflows, and tool-augmented assistants.
How deep it goes
Defines prompt chaining and explains its modular benefits with an example, without detailing implementation.
Examples
Illustrates chaining by extracting relevant document quotes before generating an answer.
What you can do after
Understand how to divide a task across prompts so one step’s output informs the next.

Read how Introduction to Generative AI, Second Edition explains pipeline →

Starting Data Analytics with Generative AI and Python

A data pipeline is a sequence of processes that moves and transforms data between sources, storage, and analytical stages.

How it is used
Frames data analysis as a decision-support process rather than a simple transformation of raw data into charts.
How deep it goes
Defines collection, cleaning, processing, and presentation, but defers detailed methods for each stage.
Examples
Uses retail, healthcare, and manufacturing examples with distinct data sources and analytical goals.
What you can do after
Apply a staged workflow that collects, prepares, analyzes, and presents data for a defined business question.

Read how Starting Data Analytics with Generative AI and Python explains pipeline →

Microsoft Azure in Action

A pipeline is an ordered sequence of processing stages that moves data or executes computational tasks.

How it is used
Moves command results or AI data through successive processing stages for inspection, ingestion, labeling, training, and deployment.
How deep it goes
Explains PowerShell chaining and AI data connections, but stops short of detailing complete end-to-end solution development.
Examples
Uses Get-AzStorageBlob piped to select Name and nightly review files flowing from Blob Storage into AI workflows.
What you can do after
Apply it via PowerShell pipelines or configure AI Foundry storage connections, permissions, and downstream data-processing steps.

Read how Microsoft Azure in Action explains pipeline →

Deep Learning with PyTorch, Second Edition

How it is used
Packages generation components behind an image-mask-prompt call and partitions models into stages for multi-GPU training.
How deep it goes
Explains component bundling and stage scheduling through examples, while stopping short of inpainting internals.
Examples
Uses horse-to-zebra inpainting, GPipe microbatches, historical NLP workflows, and a proposed segmentation-classification chain.
What you can do after
Apply it through Diffusers to inpaint an image, or configure PyTorch pipeline stages and a GPipe schedule.

Read how Deep Learning with PyTorch, Second Edition explains pipeline →

100 C++ Mistakes and How to Avoid Them

How it is used
Uses C++ ranges to compose value-stream operations, feeding each stage’s output into the next.
How deep it goes
Defines mapping and filtering and demonstrates their composition with accumulation, without detailing broader range internals.
Examples
Uses integers transformed to absolute values, filtered for evenness, then accumulated.
What you can do after
Apply it via C++ ranges and views to compose transformations and predicates over values.

Read how 100 C++ Mistakes and How to Avoid Them explains pipeline →

Quarkus in Action

A reactive pipeline is a composition of asynchronous processing stages connected so that results, completion, and failures flow between them.

How it is used
Uses asynchronous continuations to connect processing stages in Quarkus applications and reactive services.
How deep it goes
Explains stage execution, event flow, subscription, and fluent construction, but does not detail internal implementation.
Examples
Demonstrates a Mutiny Multi that transforms, observes, filters, completes, and delivers selected strings to a subscriber.
What you can do after
Build reactive pipelines with Mutiny's Uni and Multi fluent APIs, then trigger them through subscription.

Read how Quarkus in Action explains pipeline →

Go in Practice, Second Edition

A template pipeline is a Go template construct that sequences commands and supports value processing and control logic.

How it is used
Formats template data and expresses iteration or conditional presentation logic without moving display concerns into application code.
How deep it goes
Explains chaining, final-argument passing, custom-function registration, and range scope, but stops short of broader template internals.
Examples
Uses date formatting, quoted printf output, transformed links, and comment iteration with current and outer-scope values.
What you can do after
Apply it via html/template to chain built-in commands and registered Go functions while formatting or controlling rendered output.

Read how Go in Practice, Second Edition explains pipeline →

Learn Go with Pocket-Sized Projects

A template pipeline is a sequence of template commands in which each command's output becomes input to the next command.

How it is used
Sequences template actions when one action’s output must become the next action’s input.
How deep it goes
Defines pipe-based composition and demonstrates function registration, but stops short of deeper execution internals.
Examples
Uses URL hostname extraction, lowercase normalization, println output, and a slice of example URLs.
What you can do after
Apply it in Go templates by registering functions with Funcs and chaining their outputs with |.

Read how Learn Go with Pocket-Sized Projects explains pipeline →

Data-Oriented Programming in Java

A pipeline is a sequence of processing stages in which data is passed from one stage to the next.

How it is used
Uses staged data transformations to explore domain behavior and guide implementation.
How deep it goes
Explains inputs, outputs, branches, and method extraction, but defers external service and repository details.
Examples
Uses late-fee processing, explicit decision variants, and automatically generated business-rule documentation.
What you can do after
Build it from scratch by mapping domain inputs to outputs, modeling alternatives explicitly, and converting settled stages into methods.

Read how Data-Oriented Programming in Java explains pipeline →

Spring in Action, Sixth Edition

A pipeline is a sequence of processing components through which data or messages flow from an input toward an output.

How it is used
Connects external systems or reactive transformations by moving messages and data through ordered processing components.
How deep it goes
Defines pipeline flow and contrasts imperative and reactive execution, but defers detailed operator behavior and configuration.
Examples
Uses a name-to-greeting transformation with uppercase conversion, mapping, subscription, and printing.
What you can do after
Build it from scratch using Spring configuration or Reactor operators to connect processing stages into a data flow.

Read how Spring in Action, Sixth Edition explains pipeline →

Rust in Action: Systems programming concepts and techniques

A pipeline is a sequence of processing stages in which each stage transforms data for the next stage.

How it is used
Uses staged transformations to turn input into an SVG file while keeping each processing responsibility separate.
How deep it goes
Explains four stages and their data flow, then shows orchestration in main(); it stops short of general pipeline abstractions.
Examples
Uses render-hex, hexadecimal operations, a virtual pen, SVG path commands, and a saved SVG document as concrete artifacts.
What you can do after
Build it from scratch by organizing a Rust program into parsing, conversion, document generation, and file-saving stages.

Read how Rust in Action: Systems programming concepts and techniques explains pipeline →

Acing the Frontend Interview

pipe is a function-composition utility that applies a sequence of functions from left to right.

How it is used
Applies sequential data transformations while assessing functional-programming reasoning in interviews.
How deep it goes
Defines left-to-right composition and sketches reduce-based implementation, but defers detailed performance optimization.
Examples
Uses API records processed through filtering, mapping, reduction, and class-based data-source and data-sink boundaries.
What you can do after
Apply functional pipelines by composing modular transformations and explaining their predictability and maintainability trade-offs.

Read how Acing the Frontend Interview explains pipeline →

Grokking Concurrency

Pipeline processing is a technique that divides computation into stages so different stages can operate concurrently on different portions of the work.

How it is used
Uses staged task decomposition to structure concurrent applications and overlap work across workers or cores.
How deep it goes
Explains consecutive stages, worker handoffs, resource overlap, and a Python implementation while deferring detailed execution mechanics.
Examples
Demonstrates laundry stages, CPU instruction stages, and a Python washer-dryer-folder program.
What you can do after
Apply via Python threads and queues to divide staged work and observe overlapping processing.

Read how Grokking Concurrency explains pipeline →

Data for All

A data pipeline is an automated or structured sequence that moves and processes data between sources and destinations.

How it is used
Supports firms’ analytics operations by supplying data to models and operational engines that predict and influence behavior.
How deep it goes
Explains dependence on data and disruption from external change, but stops short of implementation details.
Examples
Uses the COVID pandemic, social media, news, shopping, and buy-now-pay-later services as examples of pipeline dependence.
What you can do after
Evaluate how changing or withholding data can disrupt predictions and alter companies’ targeting and engagement.

Read how Data for All explains pipeline →

Architecture Modernization: Socio-technical alignment of software, strategy, and structure

A workflow is an ordered sequence of activities through which work progresses toward a result.

How it is used
Moves code toward production and helps teams inspect business or customer flows for bottlenecks, frustration, and missed outcomes.
How deep it goes
Explains automated delivery steps, environment provisioning, lead-time measurement, and workflow mapping without detailing implementation mechanics.
Examples
Uses commit-to-production delivery, automated environments, customer funnels, subscription cancellation, and support-agent handoffs as examples.
What you can do after
Operate delivery pipelines, measure change lead time, and diagnose workflow friction by mapping steps, handoffs, and dropout points.

Read how Architecture Modernization: Socio-technical alignment of software, strategy, and structure explains pipeline →

Graph Neural Networks in Action

A data pipeline is a sequence of processes that moves data from an input state through transformation and preparation stages to an output suitable for storage or computation.

How it is used
Moves raw graph data toward a preprocessed dataset suitable for GNN processing and training.
How deep it goes
Defines representations and outlines exploration, conversion, preprocessing, and splitting while leaving implementation details problem-dependent.
Examples
Uses edge lists, adjacency matrices, NetworkX graph objects, PyG data objects, masks, and graph visualizations as concrete artifacts.
What you can do after
Apply it via NetworkX and PyG to explore, convert, preprocess, and split graph data for GNN use.

Read how Graph Neural Networks in Action explains pipeline →

Build High Value Software Systems

A sequence of stages through which a software change moves during delivery or deployment.

How it is used
Keeps staging realistic by automatically refreshing it with fresh, anonymized production data.
How deep it goes
Shows a weekly automated staging-data refresh, but does not detail pipeline implementation or stages.
Examples
Uses a Monday refresh that preserves legacy schemas, high-volume users, and unusual character encodings.
What you can do after
Configure a weekly pipeline to refresh staging with anonymized production data and preserve edge cases.

Read how Build High Value Software Systems explains pipeline →

AI Agents in Action

Prompt chaining is a technique that decomposes a problem into sequential prompts whose outputs are passed to later prompts for stepwise processing.

How it is used
Structures complex problem-solving as sequential prompts that pass intermediate outputs forward.
How deep it goes
Defines the technique and works through three prompts, but stops short of evaluation and scoring.
Examples
Uses an Alex time-travel word problem and a three-stage decomposition, calculation, and solution flow.
What you can do after
Understand how to structure a problem-solving task as prompts that pass outputs from one step to the next.

Read how AI Agents in Action explains pipeline →

Context Engineering

A pipeline is a sequence of connected computational components that passes data through processing steps.

How it is used
Coordinates how context moves through sequential execution steps in LLM and agent systems.
How deep it goes
Explains phase selection, component connections, visible data flow, and error containment through stage status and confidence.
Examples
Uses evidence retrieval, answer drafting, citation validation, and a Haystack retriever-to-assembler-to-formatter flow.
What you can do after
Build explicit Haystack flows that connect retrieval, context assembly, and response formatting.

Read how Context Engineering explains pipeline →

CUDA for LLMs

A pipeline is an execution organization that overlaps successive stages of data movement and computation.

How it is used
Coordinates TMA data movement and computation through staged producer-consumer work.
How deep it goes
Specifies producer-thread counts, buffer stages, and coordinating barriers, leaving these design choices to kernel developers.
What you can do after
Configure producer threads, circular-buffer stages, and barriers for a TMA producer-consumer pipeline.

Read how CUDA for LLMs explains pipeline →

Also covered in

What to know first

Related concepts

Questions about pipeline

What is pipeline?

A pipeline is an ordered sequence of connected processing stages through which data or work flows, with each stage passing or transforming its output for subsequent stages.

Which Manning books explain pipeline?

Natural Language Processing in Action, Second Edition, Build AI-Enhanced Web Apps, AI Agents and Applications, Crack Any Codebase with AI, Machine Learning System Design, Delivering Machine Learning Projects: From design to deployment, "Looks Good to Me", Financial AI in Practice and 55 more.

What should I know before learning pipeline?

Manning books that teach pipeline usually introduce language model, Python, prompt and AI agent first.

What is pipeline part of?

Pipeline is part of data layer. A software architectural layer that mediates access to data by storing, retrieving, transforming, or exposing it to other components.

What are the parts of pipeline?

Pipeline includes add operation, audio capture and browser interface.

What is pipeline also called?

Pipeline is also called pipe.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.