liveBook · Manning

Sparse representation

As explained in AI Recommender Systems

A sparse matrix is a matrix representation that stores only nonzero or nonempty entries rather than allocating storage for every cell.

A sparse matrix stores nonzero entries and their positions instead of allocating space for every cell. In recommender systems, user-item matrices are mostly empty because each user interacts with only a small share of items. section 2.2.1 section 3.3.4

For a matrix of 162,541 users and 59,047 movies, storing every cell would take about 56 GB; storing 25 million nonzero entries takes about 95 MB. CSR also supports efficient input to similarity calculations and ALS training. section 2.2.2 section 3.3.4

Listing 2.1 Creating the user-item matrix
user_ids = ratings['userId'].unique()
movie_ids = ratings['movieId'].unique()
user_to_idx = {uid: idx for idx, uid in enumerate(user_ids)} #A
movie_to_idx = {mid: idx for idx, mid in enumerate(movie_ids)} #A
idx_to_movie = {idx: mid for mid, idx in movie_to_idx.items()} #A
 
rows = [user_to_idx[uid] for uid in ratings['userId']] #B
cols = [movie_to_idx[mid] for mid in ratings['movieId']] #B
data = [1] * len(ratings) #B
 
user_item_matrix = csr_matrix(
    (data, (rows, cols)),
    shape=(len(user_ids), len(movie_ids))
) #C

See how interaction records, row and column indices, and matrix dimensions are assembled into a CSR user-item matrix. section 2.2.2

How AI Recommender Systems covers it

How deep it goes
Defines sparsity and demonstrates CSR construction and memory costs, without detailing sparse-format internals.
Examples
Uses MovieLens user-movie interactions, comparing a 56 GB dense matrix with a 95 MB sparse one.
What it assumes
Assumes user-item interaction records and identifier mappings while defining matrix rows, columns, and interaction values.
What you can do after
Build a CSR user-item matrix with SciPy and use it in similarity or ALS workflows.
How it connects
Connects sparse matrices to user-item matrices, cosine similarity, collaborative filtering, and ALS.

Also explained in

Compare how 5 books explain sparse representation →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Recommender Systems. The section links point to the parts of the book it draws on.