Categorical encoding

As explained in Grokking Machine Learning

A data-encoding technique that represents each category of a categorical feature with a separate binary indicator feature.

One-hot encoding turns a categorical feature into numerical indicator features, making the categories usable by machine-learning models without assigning them an artificial numeric order. section 13.3 section 13.7

For each categorical feature:

• Create a separate binary feature for each category. section 9.3 section 13.3
• Mark whether each example belongs to each category; for a dog, cat, or bird feature, the dog example becomes yes, no, no. section 9.3
• Prefer these indicators to assigning categories numbers when that would imply an unintended order. section 13.3

Passenger data

Sex and port of embarkation are categorical features in the passenger dataset; one-hot encoding turns them into numerical features for a model. section 13.3

Figure 13.1 One-hot encoding our data to turn it all into numbers for the machine learning model to read. On the left, we have columns with categorical features such as gender or port of embarkment. On the right, we have turned these categorical features into numerical features.

Compare the categorical columns with their numerical indicator features. section 13.3

How Grokking Machine Learning covers it

How deep it goes

Defines one indicator column per category and explains why numeric labels can imply an unwanted order.

Examples

Uses dog, cat, and bird indicators, plus passenger-data features such as sex and embarkation port.

What it assumes

Distinguishes numerical from categorical features using basic examples before describing their conversion.

What you can do after

Apply it with Pandas `get_dummies` to turn categorical columns into numerical indicator columns.

How it connects

Connects categorical encoding to decision trees, linear models, and feature engineering.

liveBookmanning.com

Read Grokking Machine Learning on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Natural Language Processing in Action, Second EditionA one-hot encoder is a component that maps categorical items to binary vectors with one active position per item.
• 

  liveBookmanning.com
  Deep Learning with PyTorch, Second EditionAn encoding in which exactly one position is set to indicate a category and all others are zero.
• 

  liveBookmanning.com
  Learn Generative AI with PyTorchA vector representation with a single active entry used here to encode token or position identities before projection.
• 

  liveBookmanning.com
  Math and Architectures of Deep LearningA vector with exactly one element equal to 1 and all others equal to 0, used here to encode the correct class.
• 

  liveBookmanning.com
  Grokking AI Algorithms, Second EditionOne-hot encoding represents each category by a separate binary indicator, with one indicator active for each example.
• 

  liveBookmanning.com
  Ensemble Methods for Machine LearningCategorical encoding is the transformation of categorical feature values into numerical representations suitable for machine-learning models.
• 

  liveBookmanning.com
  Machine Learning System DesignOne-hot encoding is a representation technique that converts a categorical value into a vector with one active indicator for its category.
• 

  liveBookmanning.com
  The Art of AI Product DevelopmentOne-hot encoding represents each item in a categorical set as a vector with one active element and all other elements zero.
• 

  liveBookmanning.com
  Outlier Detection in PythonCategorical encoding is the transformation of categorical values into numerical representations suitable for computational analysis.
• 

  liveBookmanning.com
  Machine Learning Platform EngineeringOne-hot encoding is a technique that represents each value of a categorical variable with a separate binary indicator feature.
• 

  liveBookmanning.com
  Deep Learning with Python, Third EditionA representation scheme where one position is 1 and all others are 0 to encode a discrete category or symbol.
• 

  liveBookmanning.com
  Build a Text-to-Image Generator (from Scratch)A one-hot tensor represents each categorical value with a vector containing one active entry and zeros elsewhere.

Compare how 13 books explain categorical encoding →

Related concepts

• embeddingcontrasts with · A numerical vector representation of an item or input that encodes features or relationships useful for computational tasks.
• label encodingincludes · Label encoding is a categorical-data transformation that maps each category to a numerical code, which may imply an ordering among categories.
• preprocessingpart of · Preprocessing is the preparation or transformation of data, content, or other inputs into a form suitable for a subsequent computational operation.
• cross entropy lossuses · Cross-entropy loss measures the expected negative log probability that a predicted distribution assigns to outcomes drawn from a target distribution.
• one-hot encodingincludes · One-hot encoding represents each category as a binary vector or set of indicators with exactly one active component identifying that category.
• vectorizationproduced by · Vectorization is the representation of data or computations as vectors or arrays so they can be processed numerically or collectively.

This explainer was generated by Manning's knowledge graph from the text of Grokking Machine Learning. The section links point to the parts of the book it draws on.