Alternating least squares (ALS) is an optimization algorithm for matrix factorization that learns user and item latent factors by fixing one set of factors and solving for the other with ordinary regularized least squares, then swapping roles until the model converges. In recommender systems, ALS is one of the standard ways to turn a sparse user-item matrix into dense embeddings that predict ratings or preferences.
ALS is valued for two practical reasons: each half-step has a closed-form solution, and the per-user and per-item solves are independent and easy to parallelize. Its implicit-feedback variant became a default baseline for collaborative filtering on large clickstream and purchase data.
How alternating least squares solves matrix factorization
ALS solves matrix factorization by breaking a non-convex problem into a sequence of convex ones. The model approximates the rating matrix R as the product of a user factor matrix U and an item factor matrix V, each with k latent dimensions, and minimizes the squared error on observed entries plus an L2 regularization term weighted by lambda.
Jointly, the objective is non-convex because U and V multiply each other. When V is held fixed, however, the loss for each user vector is a ridge regression problem with an exact solution: the user vector equals the inverse of (V transposed V plus lambda I) times V transposed r. The algorithm alternates two steps:
- Fix item factors and solve a small k-by-k linear system for every user.
- Fix user factors and solve the corresponding system for every item.
- Repeat until the training loss or a validation metric stops improving, typically after 10 to 20 iterations.
Each full sweep is guaranteed not to increase the objective, so ALS converges monotonically to a local minimum or stationary point. It does not guarantee the global optimum, which is why initialization and regularization still matter.
Explicit and implicit feedback variants
ALS exists in two main forms that differ in how they treat missing entries. The explicit-feedback form fits only known ratings, while the implicit-feedback form treats every user-item pair as an observation with a confidence weight, a design popularized for implicit signals such as views, plays and purchases.
In the implicit variant, each interaction is converted into a binary preference (interacted or not) and a confidence value, commonly c = 1 + alpha times the interaction count. Unobserved pairs receive low confidence rather than being ignored. A key algebraic trick keeps this tractable: the dense term V transposed V is computed once per sweep, and only the user’s observed items add a correction.
| Property | Explicit ALS | Implicit ALS |
|---|---|---|
| Input data | Ratings such as 1-5 stars | Counts of clicks, plays or purchases |
| Missing entries | Ignored in the loss | Treated as weak negatives |
| Target | Predicted rating value | Preference score for ranking |
| Extra hyperparameter | None beyond lambda and k | Confidence scale alpha |
| Typical evaluation | RMSE or MAE | Precision at K, recall at K, NDCG |
Key hyperparameters and their effects
ALS has few hyperparameters, but each one has a direct and predictable effect on accuracy, overfitting and cost. Tuning is usually done by grid or Bayesian search against a time-based holdout set.
| Parameter | Typical range | Effect when increased | Risk |
|---|---|---|---|
| Rank k | 16-256 | Captures finer taste patterns | Overfitting and higher cost |
| Regularization lambda | 0.001-1 | Shrinks factors toward zero | Underfitting if too high |
| Confidence alpha | 1-100 | Trusts frequent interactions more | Popular items dominate |
| Iterations | 5-30 | Lowers training loss | Little gain after convergence |
Computationally, each user solve costs about O(n_u k squared + k cubed), where n_u is the number of items the user touched. The cubic dependence on k explains why ranks above a few hundred become expensive, and why conjugate-gradient solvers are often used instead of exact inversion for large ranks.
ALS compared with other factorization methods
ALS competes mainly with stochastic gradient descent (SGD) for the same matrix factorization objective, and with ranking-oriented methods such as Bayesian personalized ranking. The best choice depends on data type, cluster setup and whether the goal is rating prediction or top-N recommendation.
| Method | Optimization style | Strength | Weakness |
|---|---|---|---|
| ALS | Exact block updates | Parallel, stable, handles implicit data | Cubic cost in rank |
| SGD factorization | Per-entry gradient steps | Cheap per step, flexible losses | Sensitive learning rate, harder to parallelize |
| Bayesian personalized ranking | Pairwise gradient steps | Optimizes ranking directly | Requires negative sampling |
| Neural collaborative filtering | Deep network training | Models nonlinear interactions | Often no better than tuned factorization |
Distributed engines popularized ALS because user and item blocks can be partitioned across machines and updated without locking. Apache Spark MLlib, for example, ships a blocked ALS implementation, and several GPU libraries offer fast implicit ALS.
Using ALS in a production pipeline
In modern systems, ALS often serves as a candidate generation stage rather than the final ranker. Its item vectors support fast nearest-neighbor retrieval, and its scores feed a later learning-to-rank model in a multi-stage ranking pipeline.
- Log-scale or cap raw counts before computing confidence so a few heavy users do not dominate.
- Evaluate on a time-based split, because random splits leak future behavior into training.
- Fold in new users by solving one least squares step against fixed item factors instead of retraining.
- Retrain on a schedule, commonly daily, since ALS is batch-oriented.
Limitations and common pitfalls
ALS inherits the limits of pure collaborative filtering. It cannot score items or users without interactions, so the cold start problem requires content features or hybrid models. It also tends to amplify popularity bias, because frequently consumed items receive strong, well-estimated vectors.
Common mistakes include treating implicit data as explicit ratings, tuning only for RMSE when the product shows ranked lists, and setting alpha so high that recommendations collapse toward bestsellers. Another frequent error is forgetting to regularize, which lets users with few interactions receive extreme vectors.
A plain-language summary
Alternating least squares is a way for a recommendation engine to learn hidden tastes from what people have watched, bought or rated. It describes every person and every product with a short list of numbers, so that people and products that fit together end up with similar numbers.
The clever part is that it adjusts one side at a time. First it keeps the product descriptions fixed and improves the descriptions of people, then it does the reverse. Each round gets a little better, much like tuning two dials by turning one, then the other, until the sound is right.
An example from practice
A mid-sized online music service has about two million listeners and 500,000 tracks, but each listener has played only a few hundred tracks. The team builds a play-count matrix, applies a log transform, and trains implicit ALS with rank 128, lambda 0.05 and alpha 20 on a small cluster in under an hour each night.
The resulting track vectors feed an approximate nearest-neighbor index that proposes 500 candidates per listener, which a gradient-boosted ranker then orders using recency and context. Compared with a popularity baseline in an online test, the ALS-based pipeline raises recall at 50 substantially and increases the share of plays from personalized suggestions, while new listeners are served by a content-based fallback until they have enough history.