liveBook · Manning

Preprocessing

Also called data preparation, preprocess, pre-processing, dataset preparation

Preprocessing is the preparation or transformation of data or other inputs into a form suitable for subsequent processing, analysis, or execution.

How 25 Manning books explain it

Deep Learning with R, Third Edition

Data preprocessing is the preparation of raw data by reshaping, scaling, encoding, or otherwise converting it into a model-compatible form.

How it is used
Prepares raw inputs and targets for neural networks before model training and evaluation.
How deep it goes
Works through normalization, vectorization, and domain-specific pipelines, while deferring instruction in regular expressions.
Examples
Uses MNIST digits, JPEG image directories, movie reviews, and grid-based bounding-box targets.
What you can do after
Apply preprocessing with R, Keras, and tf.data to prepare image and text datasets.

Read how Deep Learning with R, Third Edition explains preprocessing →

Data Without Labels

Data preprocessing, or data preparation, organizes and transforms raw data into a form suitable for analysis or model training.

How it is used
Positions preprocessing between data discovery and exploratory analysis or modeling, making raw data usable for the intended task.
How deep it goes
Details common data problems and task-specific treatments, but the supplied passages do not spell out the code for each operation.
Examples
Uses retail invoices requiring conversion for association rules and text cleaned through stop-word removal, normalization, and tokenization.
What you can do after
Practice/perform data-quality inspection and choose preparation steps suited to a dataset and its modeling task.

Read how Data Without Labels explains preprocessing →

Software Testing with Generative AI

Data preprocessing is the preparation and transformation of data into a form suitable for model training or fine-tuning.

How it is used
Prepares business-domain documentation and code for instruction-based fine-tuning.
How deep it goes
Explains schemas, parsing choices, prompt encoding, and tokenization, but stops short of detailing tokenizer internals.
Examples
Uses JSONL instruction-output pairs and Java controller and test-code fragments to contrast coherent records with isolated lines.
What you can do after
Apply it via Axolotl to tokenize a prepared dataset before launching a fine-tuning run.

Read how Software Testing with Generative AI explains preprocessing →

Transfer Learning for Natural Language Processing

Data preprocessing is the preparation and transformation of raw data into a form suitable for analysis or machine learning.

How it is used
Prepares labeled NLP datasets for classification baselines and later comparisons with more sophisticated models.
How deep it goes
Walks through collection, tokenization, normalization, stop-word removal, sampling, and dataset assembly, with a sentiment-specific adjustment.
Examples
Uses Enron and fraudulent emails for spam classification and labeled IMDB reviews for sentiment classification.
What you can do after
Apply it via Python to prepare labeled email and review datasets for classification.

Read how Transfer Learning for Natural Language Processing explains preprocessing →

Optimization Algorithms: AI techniques for design, planning, and control problems

A preprocessing phase is an initial computation that transforms or augments data before subsequent queries or operations.

How it is used
Uses preprocessing to accelerate shortest-path queries by preparing a graph before search.
How deep it goes
Explains node ordering, contraction, shortcuts, and querying, with a predetermined-order Python implementation.
Examples
Illustrates contraction with road networks and compares explored-node counts on an 80-mile query.
What you can do after
Evaluate/understand how preprocessing trades preparation time for faster shortest-path queries.

Read how Optimization Algorithms: AI techniques for design, planning, and control problems explains preprocessing →

Deep Learning with Python, Third Edition

The transformation of raw data into a form suitable for input to a model.

How it is used
Prepares raw inputs and targets for neural networks, adapting their representation and scale to the model.
How deep it goes
Works through vectorization, normalization, and text preparation, while only naming missing-value handling.
Examples
Uses IMDb reviews, grayscale digits, house-price features, and grid-aligned detection targets to illustrate transformations.
What you can do after
Apply preprocessing through Python vocabulary lookups and tf.data image pipelines.

Read how Deep Learning with Python, Third Edition explains preprocessing →

Grokking Machine Learning

Data preprocessing is the transformation of raw data into a form suitable for computational analysis or model training.

How it is used
Prepares raw emails, labels, images, and tabular records for model training and evaluation.
How deep it goes
Demonstrates concrete transformations and an end-to-end workflow, while stopping short of exhaustive data-cleaning rules.
Examples
Uses spam emails, circular two-class data, MNIST images, and the Titanic dataset as preprocessing examples.
What you can do after
Apply preprocessing with Pandas and Keras by transforming text, labels, images, and tabular data into model-ready representations.

Read how Grokking Machine Learning explains preprocessing →

Ensemble Methods for Machine Learning

Preprocessing is the transformation of raw data into a representation suitable for machine-learning algorithms.

How it is used
Converts text, categorical values, and numeric measurements into representations suitable for machine-learning algorithms.
How deep it goes
Walks through cleaning, tokenization, lemmatization, vectorization, vocabulary filtering, TF-IDF, encoding, and standardization with implementation examples.
Examples
Uses movie reviews, a term-document matrix, TF-IDF extraction, categorical variables, and the AutoMPG regression data set.
What you can do after
Apply it via NLTK and scikit-learn to clean, encode, vectorize, reweight, and standardize data before training.

Read how Ensemble Methods for Machine Learning explains preprocessing →

Distributed Machine Learning Patterns

The transformation and preparation of raw data into a form suitable for downstream computation or model training.

How it is used
Prepares ingested images for model training and converts serving requests into model-compatible inputs.
How deep it goes
Demonstrates scaling, casting, caching, shuffling, decoding, resizing, and serving signatures without detailing transformation internals.
Examples
Uses MNIST normalization, a Fashion-MNIST tf.data pipeline, and JPEG-byte decoding for image inference.
What you can do after
Apply it via TensorFlow and tf.data to normalize training data and define preprocessing in a serving signature.

Read how Distributed Machine Learning Patterns explains preprocessing →

Grokking AI Algorithms, Second Edition

Data preparation is the process of cleaning, formatting, and transforming raw data into a form suitable for analysis or model development.

How it is used
Prepares raw data for model training within an iterative machine-learning workflow.
How deep it goes
Explains cleaning and transformation choices with their rationale for text, images, and categorical data.
Examples
Uses a diamond dataset with missing values and a scraped pet-blog image to illustrate preparation decisions.
What you can do after
Evaluate and understand how to choose cleaning and encoding steps for text, images, and tabular data.

Read how Grokking AI Algorithms, Second Edition explains preprocessing →

Machine Learning Platform Engineering

Data preparation is the process of cleaning, transforming, organizing, and otherwise making data suitable for model development.

How it is used
Anchors model development by converting data into a reproducible input for training, evaluation, and later production workflows.
How deep it goes
Breaks preparation into downloading, splitting, notebook exploration, and Kubeflow component construction, while deferring broader pipeline stages.
Examples
Uses ID-card detection and movie recommendation projects, with dataset download and training, test, and validation splits.
What you can do after
Build it from scratch as composable Kubeflow components assembled into an executable, reproducible preparation pipeline.

Read how Machine Learning Platform Engineering explains preprocessing →

Machine Learning System Design

Preprocessing is the transformation and preparation of data before it is supplied to a machine learning model for training or inference.

How it is used
Prepares collected data for aggregation, feature calculation, model training, or prediction.
How deep it goes
Explains filtering, aggregation, repair, joining, and modality-specific transformations, but does not detail individual algorithms.
Examples
Uses retail transactions, stock, promotions, weather, competitor prices, images, and text as preprocessing examples.
What you can do after
Build it from scratch as a staged transformation flow that produces cleaner, joined, feature-ready inputs.

Read how Machine Learning System Design explains preprocessing →

Generative AI in Action

Data preprocessing is the process of cleaning, transforming, and organizing raw data into a form suitable for analysis or model processing.

How it is used
Prepares enterprise data for LLM processing and retrieval-augmented generation.
How deep it goes
Lists preprocessing operations and pipeline considerations without giving an executable implementation.
Examples
Illustrates stop-word removal and identifies libraries for structured output and overlapping chunks.
What you can do after
Evaluate preprocessing and chunking options for a RAG pipeline, including their quality and performance trade-offs.

Read how Generative AI in Action explains preprocessing →

Starting Data Analytics with Generative AI and Python

Data preprocessing is the transformation and preparation of raw data into a form suitable for analysis or modeling.

How it is used
Prepares business data for descriptive analysis, visualization, statistical inference, and modeling.
How deep it goes
Explains quality checks, table merging, missing-value handling, type conversion, and derived variables, then stops before detailed modeling.
Examples
Uses merged order datasets, delivery-time differences, missing-value decisions, converted timestamps, summary statistics, and visualizations.
What you can do after
Apply preprocessing by validating quality, combining relevant tables, cleaning fields, and creating variables for analysis.

Read how Starting Data Analytics with Generative AI and Python explains preprocessing →

Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI

Data preprocessing is the transformation and preparation of raw data before it is supplied to a machine learning model.

How it is used
Prepares stored data and features for machine-learning model training.
How deep it goes
Provides a worked inspection and cleanup workflow, stopping short of specifying final transformations for every problematic column.
Examples
Uses a Kuala Lumpur real estate dataset, DataFrame statistics, unique-value counts, dimensions, and missing-value reports.
What you can do after
Apply DataFrame inspection to classify features, quantify missing data, and prioritize column cleanup before training.

Read how Machine Learning for Tabular Data: XGBoost, Deep Learning, and AI explains preprocessing →

Graph Neural Networks in Action

Data preprocessing is the transformation and preparation of data into a form suitable for model training or inference.

How it is used
Prepares graph-structured data for GNN training and inference through representation, splitting, sampling, batching, and loading.
How deep it goes
Defines graph conversion, splitting, sampling, and batching, while deferring deeper preprocessing detail to appendix B.
Examples
Demonstrates NetworkX-to-PyG conversion, tensor assignment, indexed splits, masks, and full-graph or batched model input.
What you can do after
Apply preprocessing via PyTorch Geometric and NetworkX to produce graph data suitable for GNN training.

Read how Graph Neural Networks in Action explains preprocessing →

Building Reliable AI Systems

The cleaning and transformation of raw data before it is supplied to a machine-learning process.

How it is used
Prepares and validates examples that teach a fine-tuned model the desired task and response style.
How deep it goes
Provides cleaning and formatting guidance with a sample JSONL record, but no preprocessing implementation.
Examples
Uses legal references and a technical-support prompt–completion record to illustrate preservation and formatting.
What you can do after
Practice cleaning and formatting fine-tuning examples while preserving domain notation and separating validation data.

Read how Building Reliable AI Systems explains preprocessing →

Outlier Detection in Python

The preparation and transformation of raw data into a form suitable for analysis or machine-learning models.

How it is used
Prepares heterogeneous tabular data for outlier detection and supports iterative detector development and evaluation.
How deep it goes
Explains cleaning, encoding, binning, scaling, imputation, feature work, and ensemble variation, but does not derive their algorithms.
Examples
Uses NaN counts, median-filled Strikeouts values, and IQR-based removal of extreme numeric records.
What you can do after
Apply it via tabular-data operations to produce detector-compatible features and construct preprocessing variants for ensembles.

Read how Outlier Detection in Python explains preprocessing →

Quantization and Fast Inference

Image preprocessing is the transformation of an image before model execution, such as resizing or normalizing its pixel values.

How it is used
Profiles image-pipeline latency to identify which stages consume the deployment budget.
How deep it goes
Measures decode, resize, and normalization separately, without detailing their implementation internals.
Examples
Uses JPEG-to-RGB decoding, 224×224 bilinear resizing, and ImageNet mean-and-standard-deviation normalization.
What you can do after
Diagnose latency by timing preprocessing stages alongside inference and end-to-end execution.

Read how Quantization and Fast Inference explains preprocessing →

Algorithms Every Programmer Should Know

Preprocessing is preparatory computation performed on input data before the main algorithmic operation begins.

How it is used
Prepares string-matching algorithms to search by computing auxiliary data before scanning the text.
How deep it goes
Works through bad-character-table construction, while quantifying LPS-array costs without detailing its construction.
Examples
Contrasts LPS-array construction with a stepwise bad-character table for the pattern “aabadc.”
What you can do after
Build it from scratch by calculating and overwriting Horspool bad-character shifts for a pattern.

Read how Algorithms Every Programmer Should Know explains preprocessing →

The Art of AI Product Development

Preprocessing is the transformation and preparation of raw data before it is supplied to a machine-learning model.

How it is used
Prepares raw data for mathematical computation by producing model-ready numerical representations.
How deep it goes
Defines numerical transformation, cleaning, and information loss, but stops short of detailing embedding implementation.
Examples
Uses one-hot word vectors, word embeddings, and numerical financial or statistical data as examples.
What you can do after
Evaluate preprocessing choices by checking model compatibility, modality expertise, and task-relevant information loss.

Read how The Art of AI Product Development explains preprocessing →

Deep Learning with PyTorch, Second Edition

Data preparation performed before model execution, here separating image and prompt encoding from the main model run.

How it is used
Prepares images, masks, prompts, and captions so pretrained or trainable models can consume them.
How deep it goes
Demonstrates concrete transformations and embedding preparation but defers detailed model-training and encoding internals.
Examples
Uses BLIP captions, SAM point prompts, CT images, masks, SegformerImageProcessor, and TensorBoard loss tracking.
What you can do after
Apply preprocessing via SegformerImageProcessor or SAM’s image-and-prompt workflow before model inference or training.

Read how Deep Learning with PyTorch, Second Edition explains preprocessing →

Knowledge Graphs and LLMs in Action

Data preparation is the process of selecting, cleaning, transforming, and organizing data for computational use.

How it is used
Prepares financial, recommendation, and knowledge-graph data for graph learning or structured analysis.
How deep it goes
Explains multi-stage identifier, tensor, feature, label, schema, and ontology transformations with implementation examples.
Examples
Uses financial transactions, illicit-node labels, movie genres, user ratings, medical documents, and ontology mappings.
What you can do after
Builds numerical graph inputs and structured knowledge-graph data through mappings, encoding, ingestion, and relationship generation.

Read how Knowledge Graphs and LLMs in Action explains preprocessing →

Effective Conversational AI

Data preparation is the process of transforming collected data into a suitable form for testing or model training.

How it is used
Prepares production-log utterances for testing and documents for retrieval-augmented generation searches.
How deep it goes
Outlines filtering, scrubbing, annotation, chunking, embedding, and storage without detailing each step’s implementation.
Examples
Uses logged utterances, PDFs, tables, semantic chunks, intent labels, and ideal response annotations as examples.
What you can do after
Apply it to prepare test sets from logs or searchable document data for RAG.

Read how Effective Conversational AI explains preprocessing →

Evolutionary Deep Learning: Genetic algorithms and neural networks

The process of cleaning, transforming, and organizing data for machine-learning analysis or training.

How it is used
Positions data preparation at the start of the AI/ML workflow before feature engineering and model development.
How deep it goes
Frames preparation as costly and automatable but defers concrete cleaning procedures to later treatment.
Examples
Uses the AutoML workflow diagram and names Azure Machine Learning, AutoKeras, and AutoTorch as automation platforms.
What you can do after
Evaluate where data preparation fits in an automated machine-learning workflow and why it matters for model performance.

Read how Evolutionary Deep Learning: Genetic algorithms and neural networks explains preprocessing →

Also covered in

What to know first

Related concepts

Questions about preprocessing

What is preprocessing?

Preprocessing is the preparation or transformation of data or other inputs into a form suitable for subsequent processing, analysis, or execution.

Which Manning books explain preprocessing?

Deep Learning with R, Third Edition, Data Without Labels, Software Testing with Generative AI, Transfer Learning for Natural Language Processing, Optimization Algorithms: AI techniques for design, planning, and control problems, Deep Learning with Python, Third Edition, Grokking Machine Learning, Ensemble Methods for Machine Learning and 17 more.

What should I know before learning preprocessing?

Manning books that teach preprocessing usually introduce language model, natural language processing, accuracy and data cleaning first.

What is preprocessing part of?

Preprocessing is part of pipeline. A pipeline is an ordered sequence of connected processing stages through which data or work flows, with each stage passing or transforming its output for subsequent stages.

What are the parts of preprocessing?

Preprocessing includes punctuation removal, stop-word removal, data cleaning and categorical encoding.

What is preprocessing also called?

Preprocessing is also called data preparation, preprocess, pre-processing or dataset preparation.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.