How can raw emails, labels, or images become usable by a model? Preprocessing transforms data into a form computational methods can analyze, such as word lists, categorical labels, or feature vectors. section 8.3 section 10.3
Typical preprocessing steps include:
- Lowercase email text, split it into words, remove repeated words with a set, and store the result in a new DataFrame column. section 8.3
- Convert binary class labels such as 0 and 1 into categorical vectors such as [1,0] and [0,1]. section 10.3
- Reshape rectangular images into long feature vectors and convert digit labels into vectors with one nonzero class entry. section 10.3
- Clean data, engineer features, and prepare training, validation, and testing sets before model selection and evaluation. section 13.0
Spam detection
An email dataset contains message text and a spam label. Processing each message into unique lowercase words creates features that a naive Bayes model can count by class; a phrase such as “buy cheap lottery easy money now” then receives a high spam probability. section 8.3
def process_email(text): text = text.lower() return list(set(text.split()))
Inspect the compact function that lowercases email text, splits it into words, and removes repetition before modeling. section 8.3
