Preprocessing transforms raw data into a numerical representation that an AI model can consume. It may also include cleaning data without changing its modality. section 3.1.1
One-hot encoding
In natural-language processing, one-hot encoding represents each word as a vector containing zeros and one identifying value. This is simple but coarse, so useful information may be lost; more informative word embeddings are introduced later. section 3.1.1
Treat preprocessing as a strategic modeling decision:
- Convert non-numerical modalities into internal numerical representations before feeding them to a model. section 3.1.1
- Check which task-relevant information the transformation discards. section 3.1.1
- Plan modality-specific ramp-up when the team’s existing technical skills do not match the data. section 3.1.1
Figure 3.4 One-hot encoding provides a simple numerical (algebraic) representation of words.
Use the one-hot encoding figure to inspect how words become simple numerical vectors. section 3.1.1
