Preprocessing converts raw, heterogeneous data into compact numerical structures that computational models can consume. It preserves domain meaning while organizing entities, relationships, features, and labels for learning. section 12.1.2 section 2.3
A practical preprocessing pipeline commonly includes:
- Map original entity identifiers to consecutive numeric IDs and remove records whose endpoints cannot be mapped. section 12.1.2
- Represent relationships as edge-index tensors and align feature rows with the new entity IDs. section 12.1.2
- Encode categorical information numerically, using multi-hot genre vectors or label-encoded classes. section 12.1.2 section 12.2.2
- For knowledge graphs, define schemas, process documents, disambiguate entities, map ontologies, and generate relationships. section 7.6
User–movie ratings graph
Convert movie genres into a 9,742-by-20 feature tensor, map user and movie IDs to consecutive values, and build an edge_index tensor whose columns represent ratings. section 12.2.2
Listing 12.14 Transforming genre information into a vector of features
movies_df = pd.read_csv(movies_path, index_col='movieId') #1
genres = movies_df['genres'].str.get_dummies('|') #2
movie_feat = torch.from_numpy(genres.values).to(torch.float) #3
assert movie_feat.size() == (9742, 20) #4
Inspect the pandas and PyTorch transformation that turns pipe-separated genres into numerical features. section 12.2.2
