Categorical encoding turns an identity into numbers a neural network can accept. In one-hot encoding, a vector has one position for each possible category: the chosen position is 1 and all others are 0. Red, green, and blue become [1, 0, 0], [0, 1, 0], and [0, 0, 1]. Unlike assigning ordered numbers to colors, this does not imply that one color ranks above another. section 3.5.1
import torch def onehot_encoder(position,depth): onehot=torch.zeros((depth,)) onehot[position]=1 return onehot
A small PyTorch encoder starts with a zero vector of length depth and sets the entry at position to 1. section 3.5.1
The vector length must match the number of possible identities. For integers from 0 through 99, use 100 entries: encoding 75 sets the entry at index 75. To recover a readable integer, use torch.argmax to find the active index. Applied along the last dimension, the same operation converts a batch of vectors back to numbers. section 3.5.1 section 3.5.2
Generate multiples of five
Generate ten training integers by sampling values from 0 through 19 and multiplying by 5. Encode each result as a 100-entry vector before feeding it to a GAN; the discriminator consequently takes 100 input features. After generation, argmax turns each output vector into an integer that can be checked for the desired pattern. section 3.5.2
The same representation can carry a condition. For images with or without eyeglasses, use [1, 0] or [0, 1]. Concatenate the two-entry label with a 100-entry noise vector to give a conditional GAN generator 102 inputs. For the critic, expand the label across two image channels and attach them to the three color channels. Three hair-color choices would instead need three label entries and three added channels. section 5.3.2 section 5.4.1
One-hot vectors can also represent word tokens or musical pitches, but their size matters. A vocabulary of 12,778 tokens gives every token a 12,778-entry vector; a 128-entry learned word embedding is much more compact and can represent semantic relationships that one-hot vectors do not. For music, each pitch occupies one of 84 positions in a (4, 2, 16, 84) song tensor. That example uses −1 for inactive positions and 1 for the active position to center the data around zero. section 8.2.2 section 13.1.3
