Normalization
Normalization is the process of transforming a value, representation, or data structure to satisfy a specified standard, scale, or structural constraint.
Grokking Machine Learning
A mathematical operation that rescales nonnegative quantities by their sum so that the resulting quantities sum to one.
- How it is used
- Uses normalization to convert joint spam and ham probabilities into conditional probabilities after observing a word.
- How deep it goes
- Defines normalization, explains ratio preservation, gives the divide-by-sum formula, and stops short of broader numerical methods.
- Examples
- Uses a spam-detection tree in which emails containing “lottery” yield normalized spam and ham probabilities of 3/4 and 1/4.
- What you can do after
- Apply it via division to turn compatible probability weights into values that sum to one.
How Large Language Models Work
Text normalization is the transformation of text into a standardized representation, such as by harmonizing character case or encoding.
- How it is used
- Prepares text for tokenization by reducing irrelevant variation, controlling vocabulary size, and addressing Unicode-related risks.
- How deep it goes
- Explains transformations, vocabulary effects, semantic losses, and Unicode risks, but stops short of implementation details for a complete normalizer.
- Examples
- Uses lowercasing, punctuation removal, mixed-case “LLMs,” capitalization-sensitive “Bill,” homoglyphs, and zero-width spaces.
- What you can do after
- Evaluate normalization choices by weighing smaller vocabularies and safer parsing against lost capitalization and other meaning.
Read how How Large Language Models Work explains normalization →
Classic Computer Science Problems in Swift: Essential techniques for practicing programmers
- How it is used
- Prepares attributes on different scales for k-means clustering and neural network classification.
- How deep it goes
- Defines two formulas and implements column-wise Swift transformations, but leaves constant-column handling out of the feature-scaling function.
- Examples
- Uses k-means point dimensions, an in-place Swift scaling function, and Iris and wine classification datasets.
- What you can do after
- Build it from scratch with column-wise z-scores or in-place feature scaling before clustering or classification.
Deep Learning with Python, Third Edition
The process of rescaling or normalizing features so they have more stable numerical properties for learning.
- How it is used
- Makes input features with differing ranges easier for neural networks to learn from.
- How deep it goes
- Works through feature-wise scaling and training-only statistics, then distinguishes preprocessing from adaptive batch normalization without claiming a settled mechanism.
- Examples
- Uses grayscale pixels, house-price features and targets, and differently scaled weather measurements.
- What you can do after
- Apply it via NumPy to scale inputs with training-derived statistics and convert scaled regression predictions back to their original units.
Read how Deep Learning with Python, Third Edition explains normalization →
AI-Powered Search
Normalization is the transformation of numerical features to a comparable scale, often by centering and scaling them according to their mean and standard deviation.
- How it is used
- Prepares search features for SVMrank and linear learning-to-rank models.
- How deep it goes
- Explains mean-standard-deviation scaling, training statistics, and deployment, then stops short of alternative normalization schemes.
- Examples
- Uses movie release years, title and overview BM25 scores, graded documents, and Harry Potter reranking.
- What you can do after
- Apply it via stored feature statistics and a search-engine linear ranking model before reranking candidate documents.
The Quick Python Book, Fourth Edition
Normalization is the process of converting data into a consistent canonical representation.
- How it is used
- Prepares text for word counts and phone numbers for consistent formatting.
- How deep it goes
- Works through cleaning steps and phone-number validation, while noting incomplete punctuation handling and limits of separator-pattern matching.
- Examples
- Uses a novel excerpt cleaned into words and phone numbers supplied with spaces, dots, hyphens, or parentheses.
- What you can do after
- Practice/perform text cleaning before counting words, or build a Python function that validates and reformats phone numbers.
Read how The Quick Python Book, Fourth Edition explains normalization →
Deep Learning with PyTorch, Second Edition
- How it is used
- Uses normalization to put model inputs and image channels on scales that support learning.
- How deep it goes
- Works through rescaling and mean-and-standard-deviation normalization, but leaves comparisons with custom and nonlinear alternatives as exercises.
- Examples
- Demonstrates temperature-input rescaling, column-wise chemical-data standardization, and a CIFAR-10 image transform.
- What you can do after
- Apply it via PyTorch tensor operations or `transforms.Normalize` using calculated feature or channel statistics.
Read how Deep Learning with PyTorch, Second Edition explains normalization →
Inside AI
Normalization is the transformation of numeric values to a specified bounded range so that they can be processed consistently by a model.
- How it is used
- Supports consistent model processing and equitable treatment when numeric data uses different ranges or measurement scales.
- How deep it goes
- Defines a min-max formula and works through seven values, but does not discuss alternative normalization methods or implementation details.
- Examples
- Uses seven numbers ranging from 2 to 62 and contrasts percentages with scores from 1 to 10.
- What you can do after
- Apply min-max normalization to numeric data using observed minimum and maximum values.
LLMs in Production
Normalization is the transformation of data values to a common scale or standardized representation.
- How it is used
- Prepares feature distributions for model training and contributes to the architecture of Llama 3-style models.
- How deep it goes
- Explains normalization as a high-level preparation step but stops short of pipeline implementation or mathematical detail.
- Examples
- Uses feature-distribution checks in dataset preparation and contrasts Llama 3 with a simple feed-forward network.
- What you can do after
- Apply it by aligning feature distributions with the training objective before tokenization, embedding, and model training.
Delivering Machine Learning Projects: From design to deployment
Normalization is the transformation of data values into a consistent scale, unit, or representation so that they can be compared or combined appropriately.
- How it is used
- Uses normalization to validate data integrity and make cross-market or cross-test comparisons more comparable.
- How deep it goes
- Explains normalization through data checks and comparison cautions, but stops short of specifying transformation formulas or implementation details.
- Examples
- Uses aggregate sales, published revenues, multi-country populations, economies, demographics, and epidemic or vaccine comparisons.
- What you can do after
- Diagnose/debug with it by checking normalized aggregates against business figures and questioning unsafe direct comparisons.
Read how Delivering Machine Learning Projects: From design to deployment explains normalization →
Generative AI in Action
Normalization is the transformation of data into a consistent scale, representation, or format.
- How it is used
- Supports LLM data preparation, retriever pipelines, and task-specific fine-tuning.
- How deep it goes
- Explains normalization procedurally through cleansing, tokenization, preprocessing planning, and information-retention goals.
- Examples
- Uses tokenization in fine-tuning data preparation and pairs normalization with noise removal or dimensionality reduction in retrieval preprocessing.
- What you can do after
- Apply normalization when preparing LLM, retrieval, or fine-tuning data for consistent downstream processing.
The Complete Obsolete Guide to Generative AI
Data normalization is the transformation of values to a comparable scale or representation so differences among variables can be meaningfully visualized or analyzed.
- How it is used
- Supports time-series visualization when hardware measures use incompatible units and scales.
- How deep it goes
- Explains min–max scaling with its formula and demonstrates MinMaxScaler plotting, while noting the resulting comparison limit.
- Examples
- Uses CPU clock speed, maximum RAM, and drive capacity across years in normalized and unnormalized line charts.
- What you can do after
- Apply it via MinMaxScaler to plot differently scaled data series and inspect their relative trends.
Read how The Complete Obsolete Guide to Generative AI explains normalization →
Building Web APIs with ASP.NET Core
Normalization is the organization of relational data into structured tables to reduce redundancy and preserve data integrity.
- How it is used
- Contrasts relational table design with embedded document design when choosing how board-game data and relationships should be stored.
- How deep it goes
- Explains primary keys, reference columns, fixed schemas, and junction tables, but stops short of normalization forms or procedural rules.
- Examples
- Uses BoardGames, Comments, Domains, Mechanics, and junction tables as concrete schema examples.
- What you can do after
- Evaluate whether related data belongs in normalized tables or embedded documents for a given application design.
Read how Building Web APIs with ASP.NET Core explains normalization →
Modern C
Normalization is the process of reducing a rational representation so its numerator and denominator have no unnecessary common factors.
- How it is used
- Maintains rational-number representations in reduced form and normalizes results of arithmetic.
- How deep it goes
- Defines reduction and shows a gcd-based implementation, without elaborating on gcd itself.
- Examples
- Shows reducing both fields by their gcd and normalizing product and sum results.
- What you can do after
- Apply normalization with rat_get_normal to reduce a rational value’s numerator and denominator.
Pandas Workout: 200 exercises to make you a stronger data analyst
Normalization is the organization of data into separate related structures so that each piece of information is stored only once.
- How it is used
- Cleans inconsistent values and organizes related data for flexible analysis.
- How deep it goes
- Works through value mapping and indexed joins, but stops short of formal normalization theories.
- Examples
- Uses inconsistent country and vehicle-color labels, plus separated product and sales tables.
- What you can do after
- Apply it via pandas by mapping alternate values and joining normalized data frames for analysis.
Read how Pandas Workout: 200 exercises to make you a stronger data analyst explains normalization →
Data Without Labels
Normalization is the transformation of data values into a common representation or scale so variables can be compared.
- How it is used
- Prepares comparable variables for PCA, SVD, visualization, feature extraction, and other dimensionality-reduction workflows.
- How deep it goes
- Defines normalization and places it in the PCA workflow, but defers detailed normalization methods to Python implementation and an appendix.
- Examples
- Uses house-price variables, linear combinations, covariance matrices, and successive principal components to illustrate normalized-data processing.
- What you can do after
- Apply it via Python as preprocessing for PCA and dimensionality-reduction workflows involving numerical, comparable variables.
Modern C, Third Edition
Normalization is the process of choosing a canonical representation of a rational number by reducing common factors from its numerator and denominator.
- How it is used
- Maintains rational-number representations and reduces arithmetic results to simpler forms.
- How deep it goes
- Implements reduction by computing a greatest common divisor and dividing the numerator and denominator.
- Examples
- Uses 6/15 and 2/5 as equivalent representations and normalizes multiplication and addition results.
- What you can do after
- Apply it via rat_get_normal to reduce a rational number's numerator and denominator by their greatest common divisor.
Grokking Relational Database Design
Normalization is the process of organizing relational data into separate tables to reduce redundancy and prevent update anomalies.
- How it is used
- Guides relational database design after data modeling and before converting the design into SQL tables.
- How deep it goes
- Defines the goal, normal-form sequence, key terms, and BCNF characteristics, while deferring detailed definitions to subsequent treatment.
- Examples
- Uses The Sci-Fi Collective database and separates a mixed product_review table into product and review tables.
- What you can do after
- Evaluate/understand whether a relational design needs decomposition and use normal forms to guide more consistent table structures.
Read how Grokking Relational Database Design explains normalization →
Grokking AI Algorithms, Second Edition
Normalization is the standardization of textual representation, including characters, casing, whitespace, and related formatting.
- How it is used
- Standardizes training text before tokenization so equivalent content is represented consistently.
- How deep it goes
- Explains several cleanup operations and their trade-offs, but stops short of implementation details for accented characters.
- Examples
- Uses “cafe” versus “CAFÉ,” smart quotes, repeated whitespace, and casing distinctions such as “apple” versus “Apple.”
- What you can do after
- Apply normalization to make training text uniform while choosing deliberately whether to preserve meaningful casing.
Read how Grokking AI Algorithms, Second Edition explains normalization →
Microsoft Azure in Action
Normalization is the organization of relational data into related structures to minimize redundancy and maintain each fact in one place.
- How it is used
- Organizes relational data to reduce redundancy, preserve integrity, and support maintainable updates.
- How deep it goes
- Explains table decomposition, relationships, and trade-offs but stops short of named normal forms or detailed implementation rules.
- Examples
- Uses Customers and Orders for normalized storage, then contrasts Posts and Comments with a denormalized comment count.
- What you can do after
- Evaluate relational schema choices by weighing integrity, storage, query, write, maintenance, scalability, and flexibility trade-offs.
Machine Learning System Design
The transformation of numeric data so that its values conform to a specified scale or distribution.
- How it is used
- Places numeric-feature scaling in preprocessing pipelines for forecasting and other machine-learning workflows.
- How deep it goes
- Defines standard-range scaling and contrasts model suitability, but supplies no normalization formula or implementation procedure.
- Examples
- Uses a retail forecasting pipeline and contrasts normalization with logarithms, lags, aggregates, and DeepETA tokenization.
- What you can do after
- Apply it via a preprocessing pipeline to scale numeric features while checking whether the selected model benefits from the transformation.
Read how Machine Learning System Design explains normalization →
Related concepts
- lexemeproduces · A lexeme is a normalized linguistic term produced from text for representation and matching in full-text search.
- denormalizationcontrasts with · Denormalization is the deliberate combination or duplication of related data to simplify access, support particular access patterns, or improve performance.
- redundancyaddresses · Redundancy is the presence of duplicate or overlapping information, components, or resources, whether intentional or unnecessary.
- stemminguses · Stemming is a text-normalization technique that heuristically reduces inflected or derived word forms to a shared stem, which may not be a valid word.
- min-max normalizationincludes · Min-max normalization is a feature-scaling technique that linearly maps numerical values to a specified interval using their minimum and maximum values.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.