Categorical encoding
Categorical encoding transforms discrete category values into numerical representations, such as separate binary indicators or vectors with one active position per category.
Natural Language Processing in Action, Second Edition
A one-hot encoder is a component that maps categorical items to binary vectors with one active position per item.
- How it is used
- Turns tokens into numerical vectors for NLP processing and provides a representation for skip-gram word pairs.
- How deep it goes
- Works through matrix shape, row interpretation, and NumPy construction, then gives a PyTorch encoder and notes vocabulary-mapping consistency.
- Examples
- Uses an 18-token, 15-word matrix and a PyTorch function that activates the position given by a token ID.
- What you can do after
- Build it from scratch with a fixed vocabulary and numerical vectors, and inspect the resulting token matrix.
Read how Natural Language Processing in Action, Second Edition explains categorical encoding →
Deep Learning with PyTorch, Second Edition
An encoding in which exactly one position is set to indicate a category and all others are zero.
- How it is used
- Represents categorical tensor inputs and ideal classifier targets without assigning meaningful numerical distances to labels.
- How deep it goes
- Works through tensor allocation, index selection, scatter_ and word lookup; contrasts wide one-hot vectors with embeddings.
- Examples
- Uses wine ratings, hourly weather, ASCII characters, word-vocabulary tensors and bird-versus-airplane targets.
- What you can do after
- Build it from scratch with PyTorch zero-filled tensors, scatter_ or vocabulary-index assignments.
Read how Deep Learning with PyTorch, Second Edition explains categorical encoding →
Learn Generative AI with PyTorch
A vector representation with a single active entry used here to encode token or position identities before projection.
- How it is used
- Converts category identities into neural-network inputs for number generation, conditional image generation, text, and music.
- How deep it goes
- Works through encoding, argmax decoding, and input shapes; contrasts word embeddings without detailing how their vectors are learned.
- Examples
- Uses house colors, integers from 0 through 99, eyeglasses labels, word tokens, and musical pitches.
- What you can do after
- Build it from scratch with PyTorch tensors, decode vectors with argmax, and attach categorical labels to conditional GAN inputs.
Read how Learn Generative AI with PyTorch explains categorical encoding →
Grokking Machine Learning
A data-encoding technique that represents each category of a categorical feature with a separate binary indicator feature.
- How it is used
- Converts categorical data into numerical features for machine-learning models and binary questions in decision trees.
- How deep it goes
- Defines one indicator column per category and explains why numeric labels can imply an unwanted order.
- Examples
- Uses dog, cat, and bird indicators, plus passenger-data features such as sex and embarkation port.
- What you can do after
- Apply it with Pandas `get_dummies` to turn categorical columns into numerical indicator columns.
Read how Grokking Machine Learning explains categorical encoding →
Math and Architectures of Deep Learning
A vector with exactly one element equal to 1 and all others equal to 0, used here to encode the correct class.
- How it is used
- Encodes categorical outcomes and supplies classifier ground truth for cross-entropy training.
- How deep it goes
- Defines the vector, derives its probability role, and connects it to cross-entropy and PyTorch code.
- Examples
- Uses cat, dog, airplane, and automobile labels, numeric prediction vectors, loss values, and PyTorch listings.
- What you can do after
- Apply it via PyTorch to represent class targets and compute categorical or softmax cross-entropy loss.
Read how Math and Architectures of Deep Learning explains categorical encoding →
Grokking AI Algorithms, Second Edition
One-hot encoding represents each category by a separate binary indicator, with one indicator active for each example.
- How it is used
- Prepares unordered categories as numeric inputs for statistical models and artificial neural networks.
- How deep it goes
- Defines binary indicators and works through category-to-input examples, but provides no encoding code.
- Examples
- Illustrates five diamond Cut indicators and four block-state indicators repeated across eight adjacent blocks.
- What you can do after
- Apply it to encode unordered categories and determine the corresponding number of ANN inputs.
Read how Grokking AI Algorithms, Second Edition explains categorical encoding →
Ensemble Methods for Machine Learning
Categorical encoding is the transformation of categorical feature values into numerical representations suitable for machine-learning models.
- How it is used
- Prepares categorical features for machine-learning algorithms that require numeric inputs.
- How deep it goes
- Explains ordinal and one-hot representations, demonstrates encoded matrices, and connects them to reusable preprocessing pipelines.
- Examples
- Uses color, size, Adult income data, missing workclass values, and scikit-learn encoded matrices.
- What you can do after
- Apply encoding via scikit-learn pipelines and compare encoders within ensemble-model evaluation.
Read how Ensemble Methods for Machine Learning explains categorical encoding →
Machine Learning System Design
One-hot encoding is a representation technique that converts a categorical value into a vector with one active indicator for its category.
- How it is used
- Uses one-hot encoding as basic feature engineering within model training and inference pipelines.
- How deep it goes
- Compares encoding methods and flags leakage risk, but does not detail vector construction or implementation steps.
- Examples
- Uses logistic regression preprocessing and Uber’s DeepETA pipeline as concrete examples.
- What you can do after
- Apply it within a model pipeline and select alternatives while avoiding target leakage.
Read how Machine Learning System Design explains categorical encoding →
The Art of AI Product Development
One-hot encoding represents each item in a categorical set as a vector with one active element and all other elements zero.
- How it is used
- Represents categorical data numerically before feeding it into an AI model.
- How deep it goes
- Defines the vector representation and its information loss, then defers richer word representations to word embeddings.
- Examples
- Uses words represented as zero vectors with one active element and illustrates the mapping in Figure 3.4.
- What you can do after
- Apply it to represent categorical items numerically while recognizing its coarse treatment of information.
Read how The Art of AI Product Development explains categorical encoding →
Outlier Detection in Python
Categorical encoding is the transformation of categorical values into numerical representations suitable for computational analysis.
- How it is used
- Prepares categorical features for numeric outlier detectors and other methods requiring strictly numeric predictors.
- How deep it goes
- Defines ordinal, count, and one-hot methods with concrete mappings, but defers fuller method discussion.
- Examples
- Uses color mappings and baseball position indicators to demonstrate ordinal, count, and one-hot representations.
- What you can do after
- Apply it via pandas get_dummies to convert categorical dataframe columns into binary features.
Read how Outlier Detection in Python explains categorical encoding →
Machine Learning Platform Engineering
One-hot encoding is a technique that represents each value of a categorical variable with a separate binary indicator feature.
- How it is used
- Prepares categorical features and binary targets for model training, inference, experiment tracking, reproducibility, and debugging.
- How deep it goes
- Explains implementation through encoder settings, dummy generation, column persistence, and train-test splitting, but defers encoding theory.
- Examples
- Uses an income-classification dataset with categorical features and <=50K or >50K target labels.
- What you can do after
- Builds an encoded train-test dataset and preserves feature columns for later inference.
Read how Machine Learning Platform Engineering explains categorical encoding →
Deep Learning with Python, Third Edition
A representation scheme where one position is 1 and all others are 0 to encode a discrete category or symbol.
- How it is used
- Represents categorical labels and token IDs as numeric inputs for classification and recurrent neural-network models.
- How deep it goes
- Defines the representation, implements it with NumPy and Keras, and demonstrates its dimensional cost before deferring to word embeddings.
- Examples
- Uses 46-class label vectors, 30,000-word token vectors, NumPy and Keras implementations, and a bidirectional LSTM sequence model.
- What you can do after
- Build one-hot labels or token inputs with NumPy or Keras and select a compatible categorical loss.
Read how Deep Learning with Python, Third Edition explains categorical encoding →
Build a Text-to-Image Generator (from Scratch)
A one-hot tensor represents each categorical value with a vector containing one active entry and zeros elsewhere.
- How it is used
- Conditions diffusion-model image generation on discrete class labels.
- How deep it goes
- Implements conversion, masking, embedding, and output shapes, stopping short of encoding theory.
- Examples
- Uses clothing labels 2, 9, and 0 across ten classes, producing 512- and 256-channel embedding tensors.
- What you can do after
- Builds one-hot label conditioning with PyTorch embedding blocks and configurable context masking.
Read how Build a Text-to-Image Generator (from Scratch) explains categorical encoding →
Related concepts
- embeddingcontrasts with · A numerical vector representation of an item or input that encodes features or relationships useful for computational tasks.
- label encodingincludes · Label encoding is a categorical-data transformation that maps each category to a numerical code, which may imply an ordering among categories.
- preprocessingpart of · Preprocessing is the preparation or transformation of data, content, or other inputs into a form suitable for a subsequent computational operation.
- cross entropy lossuses · Cross-entropy loss measures the expected negative log probability that a predicted distribution assigns to outcomes drawn from a target distribution.
- one-hot encodingincludes · One-hot encoding represents each category as a binary vector or set of indicators with exactly one active component identifying that category.
- vectorizationproduced by · Vectorization is the representation of data or computations as vectors or arrays so they can be processed numerically or collectively.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.