Batching
Batching is the grouping of multiple data items, requests, or operations so they can be processed together as a unit.
Kafka for Architects
Batch processing is the execution of accumulated data-processing work as a group rather than continuously as individual records arrive.
- How it is used
- Frames a choice between scheduled business processing and continuous event processing, while distinguishing that choice from Kafka’s transfer optimization.
- How deep it goes
- Explains processing windows, Kafka batch-job obstacles, and producer and consumer tuning; stops short of implementing a complete batch job.
- Examples
- Uses a daily transaction upload to a data warehouse, alongside nightly reconciliation, reporting, and batch scoring.
- What you can do after
- Evaluate/understand when scheduled processing fits, and operate/configure Kafka transport batching with throughput, latency, and memory constraints in mind.
Apache Kafka in Action
A batch is a group of Kafka records collected into one unit for transmission or storage.
- How it is used
- Groups producer records to reduce network requests and improve Kafka throughput.
- How deep it goes
- Explains buffering, partition-based batches, send triggers, configuration trade-offs, and log metadata; leaves settings to production-like performance tests.
- Examples
- Compares producer performance-test results for 10 KB records before and after setting batch.size=100000 and linger.ms=100.
- What you can do after
- Apply it via kafka-producer-perf-test.sh to compare throughput and latency before choosing batch.size and linger.ms for production-like workloads.
Graph Neural Networks in Action
Mini-batch training is a model-training technique that processes subsets of examples or graph nodes in separate batches rather than the entire dataset at once.
- How it is used
- Reduces graph-training memory demands by feeding sampled portions of data through training instead of the whole graph at once.
- How deep it goes
- Distinguishes batching from sampling and implements NeighborLoader in a GAT loop, while leaving sampler choice to experimentation.
- Examples
- Uses a 128-node NeighborLoader GAT batch and a growing geographic graph to illustrate configuration and scalability choices.
- What you can do after
- Apply it via PyTorch Geometric NeighborLoader to train a GAT in batches and test sampling choices against memory and training-time constraints.
Read how Graph Neural Networks in Action explains batching →
Deep Learning with R, Third Edition
A batch, or mini-batch, is a subset of training examples processed together in one model evaluation and parameter update.
- How it is used
- Groups training samples so a model processes them together and makes one parameter update per mini-batch.
- How deep it goes
- Defines the batch axis, walks through a gradient update, and implements an epoch loop; does not develop batch-size tuning.
- Examples
- Uses 128-image MNIST batches and generated temperature-forecasting windows batched in groups of 256.
- What you can do after
- Build it from scratch: iterate over image-and-label batches, run a training step for each, and repeat across epochs.
Read how Deep Learning with R, Third Edition explains batching →
Building Reliable AI Systems
Temporarily grouping incoming application requests to manage concurrency and reduce processing overhead.
- How it is used
- Groups incoming LLM requests to manage concurrency and traffic surges.
- How deep it goes
- Works through an asyncio queue and periodic processor, while distinguishing separate per-query calls from offline Batch API jobs.
- Examples
- Uses an asyncio queue, a two-request customer-support processor, and a Batch API submission with a 24-hour completion window.
- What you can do after
- Build it from scratch: queue interactive requests, dispatch bounded groups, and choose offline processing when longer turnaround is acceptable.
Natural Language Processing in Action, Second Edition
Batch learning is a training technique that computes parameter updates from a selected group of training examples at a time.
- How it is used
- Uses grouped training examples to balance noisy individual updates with more substantial optimization progress.
- How deep it goes
- Explains batch-size tradeoffs and gradient behavior, then stops short of detailing backpropagation internals.
- Examples
- Uses ski-slope intuition, percentage-based dataset subsets, and a PyTorch neural-network training loop.
- What you can do after
- Apply it via PyTorch by organizing forward passes, loss calculations, and optimizer updates around training data.
Read how Natural Language Processing in Action, Second Edition explains batching →
Build a Text-to-Image Generator (from Scratch)
A batch is a group of training examples processed together in one model update or evaluation step.
- How it is used
- Uses grouped examples to feed transformer, vision, diffusion, and multimodal training pipelines efficiently.
- How deep it goes
- Explains tensor conversion, grouping, shuffling, padding, masks, and batch iteration, but does not develop batch-size optimization theory.
- Examples
- Uses translation sequences, Fashion-MNIST images, clothing-item diffusion data, and paired image-caption batches.
- What you can do after
- Operate PyTorch and TorchVision data pipelines that transform, shuffle, pad, mask, and iterate through training batches.
Read how Build a Text-to-Image Generator (from Scratch) explains batching →
LLMs in Production
A microbatch is a subdivision of a larger batch that is processed as one unit within a parallel training pipeline.
- How it is used
- Reduces GPU idle time when training a model across pipeline-parallel stages.
- How deep it goes
- Explains handoff timing, idle-percentage estimation, and memory trade-offs, but stops short of detailed scheduling implementations.
- Examples
- Compares four GPUs with one batch against four microbatches and illustrates the pipeline bubble visually.
- What you can do after
- Apply microbatching to reduce pipeline stalls while evaluating GPU count, batch size, and activation memory.
Machine Learning System Design
A model-training approach that learns from a collected batch of data rather than updating continuously per incoming example.
- How it is used
- Improves model-serving throughput and supports scheduled processing when real-time responses are unnecessary.
- How deep it goes
- Explains offline, dynamic, and size-grouped batching with latency, padding, scaling, and monitoring tradeoffs.
- Examples
- Uses dynamic API request windows, size-grouped language-model inputs, and scheduled retail demand forecasts.
- What you can do after
- Apply it via scheduled jobs or dynamic inference windows while monitoring throughput, latency, failures, resources, and prediction quality.
Machine Learning Platform Engineering
Microbatching is the processing of a small group of newly arrived inputs together for analysis or computation.
- How it is used
- Supports drift monitoring, model training, and model serving by processing multiple inputs as a unit.
- How deep it goes
- Explains batch windows, report triggers, eviction, training loaders, and adaptive serving, but does not develop batching theory.
- Examples
- Uses a 100-sample rolling monitoring window, PyTorch DataLoader batch sizes, and BentoML adaptive request batching.
- What you can do after
- Apply it via BentoML or PyTorch by configuring monitoring windows, DataLoader batch sizes, or adaptive request batching.
Read how Machine Learning Platform Engineering explains batching →
Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI
A batch is a subset of training examples processed together during an incremental learning step.
- How it is used
- Enables classical out-of-core learning when tabular datasets exceed available memory.
- How deep it goes
- Explains incremental coefficient updates, latency, repeated passes, and overfitting, but leaves batch selection to experimentation.
- Examples
- Demonstrates shuffled tabular batches with scikit-learn's SGDClassifier and reports 0.818 test accuracy.
- What you can do after
- Operate/configure out-of-core training by generating batches, updating a model incrementally, and evaluating its predictions.
Read how Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI explains batching →
The Art of Code
Batching is the technique of dividing work into groups that are processed as units rather than all at once.
- How it is used
- Handles oversized or memory-heavy workloads by processing them in resource-fitting groups.
- How deep it goes
- Explains workload partitioning, capacity tradeoffs, memory benefits, and batch sizing, but does not examine partitioning internals.
- Examples
- Uses monthly invoice generation, Guava Lists.partition, 200-item groups, paginators, database cursors, and iterators.
- What you can do after
- Apply it via Guava Lists.partition to process large Java workloads gradually within capacity limits.
Kafka Streams in Action, Second Edition
A batch is a collection of records processed or transmitted together as one unit.
- How it is used
- Uses batching to reduce Kafka producer and broker requests while transmitting records to a topic.
- How deep it goes
- Explains collection boundaries, partition placement, asynchronous sending, I/O-thread timing, and callbacks without detailing configuration internals.
- Examples
- Uses sales data, ProductTransaction objects, ProducerRecord instances, produce requests, and broker acknowledgments.
- What you can do after
- Apply it via KafkaProducer.send() to produce records asynchronously and handle acknowledgment callbacks.
Read how Kafka Streams in Action, Second Edition explains batching →
Data Pipelines with Apache Airflow, Second Edition
A batch-oriented data pipeline processes data in discrete scheduled or incrementally scheduled groups rather than continuously.
- How it is used
- Uses discrete, scheduled or incrementally scheduled intervals to organize recurring data processing workflows.
- How deep it goes
- Explains schedules, intervals, DAG dependencies, execution components, backfilling, and streaming limitations without detailing implementation code.
- Examples
- Uses daily ingestion, interval-based data buckets, and bulk website-click processing as concrete batch examples.
- What you can do after
- Operate/configure recurring data workflows by scheduling interval-based runs, organizing dependent tasks, monitoring execution, and backfilling prior intervals.
Read how Data Pipelines with Apache Airflow, Second Edition explains batching →
Financial AI in Practice
The execution of data-processing work on accumulated records in groups rather than as each record arrives.
- How it is used
- Reduces repeated API-call overhead and runs scheduled financial-data merges or cohort analyses.
- How deep it goes
- Shows batch sizes, iteration, result collection, and error handling, but does not compare batching strategies.
- Examples
- Uses daily financial-data merges, top-risk user analysis, and news-text embedding batches.
- What you can do after
- Apply it via the OpenAI API to embed text groups, or process and aggregate a selected user cohort.
Related concepts
- batch sizeconfigured by · Batch size is the number of items grouped together for processing or computation as a single batch.
- pipeline parallelismused by · Pipeline parallelism is a model-parallel technique that divides sequential computation into stages on different devices and concurrently processes multiple inputs through them.
- stochastic gradient descentused by · Stochastic gradient descent is an iterative optimization method that updates parameters using gradients estimated from individual examples or randomly selected small batches.
- ACH entryhas part · An ACH entry is a payment instruction record within an ACH batch that specifies a transaction for an account.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.