liveBook · Manning

Alternating least squares

As explained in AI Recommender Systems

Alternating least squares is a matrix-factorization algorithm that iteratively estimates latent factors for users and items by minimizing weighted reconstruction error.

Alternating least squares (ALS) is a matrix-factorization method for collaborative filtering. It learns a vector for each user and item from their interactions. A user–item dot product predicts preference; comparing item vectors helps find similar items. Because the factors come from behavior rather than descriptions, they can connect items with different content but shared audiences. section 3.7 section 3.2

A ratings matrix is mostly empty: users have not rated most items. Filling those gaps with guessed ratings makes the model fit the guesses. ALS instead fits observed ratings by repeating two least-squares steps: hold item factors fixed and update user factors, then hold user factors fixed and update item factors. Each step has a closed-form solution. The alternating process is fast and parallelizable, though it does not guarantee a global optimum. section 3.3.2

Train on MovieLens interactions

For an implementation with Python’s implicit library, retain MovieLens ratings of four or five stars as positive interactions, keep the original rating as a confidence weight, and store the user–item data in compressed sparse row (CSR) format. This conversion discards lower ratings and some information; a system with clicks, views, or purchases can use those behavioral events directly. Fit an AlternatingLeastSquares model to the sparse matrix. section 3.3.4

Listing 3.4 Training an ALS model
from implicit.als import AlternatingLeastSquares
import time
model = AlternatingLeastSquares(
  factors=50,  #A
  iterations=20,  #B
  regularization=0.01,  #C
  random_state=42  #D
)
start_time = time.time()
model.fit(user_item_matrix)  #E
elapsed = time.time() - start_time #F

The training listing shows the implicit library call and a starting configuration: 50 factors, 20 iterations, and 0.01 regularization. section 3.3.4

Tune the model against held-out recommendation quality rather than training fit alone:

  • Factors set embedding size. Too few miss nuanced tastes; too many can fit noise. Compare candidate sizes on validation data. section 3.3.8
  • Regularization limits overfitting. Increase it when training results are strong but validation results are poor; lower it when both are poor. section 3.3.8
  • Iterations set the number of alternating rounds. Improvement often plateaus after 15–20, so use validation results to decide when to stop. section 3.3.8

In a recommender pipeline, ALS can retrieve personalized candidates or score candidates retrieved elsewhere. Train and save it in a batch job, then either load it for on-demand recommendations or precompute results in a database. Precomputation reduces request-time work but serves recommendations only as fresh as the last batch. Choose a retraining cadence based on changing behavior, catalog turnover, and training cost. New users and items lack the interactions ALS needs; content-based embeddings can help with that cold start. section 3.3.3 section 3.3.9 section 3.7

How AI Recommender Systems covers it

How deep it goes
Explains alternating closed-form updates and implements library training, but does not derive the least-squares solution.
Examples
Uses MovieLens ratings converted to positive interactions, a CSR matrix, and an implicit-library training listing.
What it assumes
Builds on user–item interactions and ratings while teaching their conversion to implicit feedback and sparse storage.
What you can do after
Apply it via Python’s implicit library to train a recommender and tune its factors, regularization, and iterations on validation data.
How it connects
Connects matrix factorization and user–item embeddings to collaborative filtering, sparse matrices, cold starts, and content-based embeddings.

Also explained in

Compare how 2 books explain alternating least squares →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Recommender Systems. The section links point to the parts of the book it draws on.