Alternating least squares

Also known as: ALS

On this page 8

Alternating least squares (ALS) is an optimization algorithm for matrix factorization that learns user and item latent factors by fixing one set of factors and solving for the other with ordinary regularized least squares, then swapping roles until the model converges. In recommender systems, ALS is one of the standard ways to turn a sparse user-item matrix into dense embeddings that predict ratings or preferences.

ALS is valued for two practical reasons: each half-step has a closed-form solution, and the per-user and per-item solves are independent and easy to parallelize. Its implicit-feedback variant became a default baseline for collaborative filtering on large clickstream and purchase data.

How alternating least squares solves matrix factorization

ALS solves matrix factorization by breaking a non-convex problem into a sequence of convex ones. The model approximates the rating matrix R as the product of a user factor matrix U and an item factor matrix V, each with k latent dimensions, and minimizes the squared error on observed entries plus an L2 regularization term weighted by lambda.

Jointly, the objective is non-convex because U and V multiply each other. When V is held fixed, however, the loss for each user vector is a ridge regression problem with an exact solution: the user vector equals the inverse of (V transposed V plus lambda I) times V transposed r. The algorithm alternates two steps:

  1. Fix item factors and solve a small k-by-k linear system for every user.
  2. Fix user factors and solve the corresponding system for every item.
  3. Repeat until the training loss or a validation metric stops improving, typically after 10 to 20 iterations.

Each full sweep is guaranteed not to increase the objective, so ALS converges monotonically to a local minimum or stationary point. It does not guarantee the global optimum, which is why initialization and regularization still matter.

Explicit and implicit feedback variants

ALS exists in two main forms that differ in how they treat missing entries. The explicit-feedback form fits only known ratings, while the implicit-feedback form treats every user-item pair as an observation with a confidence weight, a design popularized for implicit signals such as views, plays and purchases.

In the implicit variant, each interaction is converted into a binary preference (interacted or not) and a confidence value, commonly c = 1 + alpha times the interaction count. Unobserved pairs receive low confidence rather than being ignored. A key algebraic trick keeps this tractable: the dense term V transposed V is computed once per sweep, and only the user’s observed items add a correction.

PropertyExplicit ALSImplicit ALS
Input dataRatings such as 1-5 starsCounts of clicks, plays or purchases
Missing entriesIgnored in the lossTreated as weak negatives
TargetPredicted rating valuePreference score for ranking
Extra hyperparameterNone beyond lambda and kConfidence scale alpha
Typical evaluationRMSE or MAEPrecision at K, recall at K, NDCG
Comparison of the two ALS formulations

Key hyperparameters and their effects

ALS has few hyperparameters, but each one has a direct and predictable effect on accuracy, overfitting and cost. Tuning is usually done by grid or Bayesian search against a time-based holdout set.

ParameterTypical rangeEffect when increasedRisk
Rank k16-256Captures finer taste patternsOverfitting and higher cost
Regularization lambda0.001-1Shrinks factors toward zeroUnderfitting if too high
Confidence alpha1-100Trusts frequent interactions morePopular items dominate
Iterations5-30Lowers training lossLittle gain after convergence
Main ALS hyperparameters

Computationally, each user solve costs about O(n_u k squared + k cubed), where n_u is the number of items the user touched. The cubic dependence on k explains why ranks above a few hundred become expensive, and why conjugate-gradient solvers are often used instead of exact inversion for large ranks.

ALS compared with other factorization methods

ALS competes mainly with stochastic gradient descent (SGD) for the same matrix factorization objective, and with ranking-oriented methods such as Bayesian personalized ranking. The best choice depends on data type, cluster setup and whether the goal is rating prediction or top-N recommendation.

MethodOptimization styleStrengthWeakness
ALSExact block updatesParallel, stable, handles implicit dataCubic cost in rank
SGD factorizationPer-entry gradient stepsCheap per step, flexible lossesSensitive learning rate, harder to parallelize
Bayesian personalized rankingPairwise gradient stepsOptimizes ranking directlyRequires negative sampling
Neural collaborative filteringDeep network trainingModels nonlinear interactionsOften no better than tuned factorization
ALS against related training approaches

Distributed engines popularized ALS because user and item blocks can be partitioned across machines and updated without locking. Apache Spark MLlib, for example, ships a blocked ALS implementation, and several GPU libraries offer fast implicit ALS.

Using ALS in a production pipeline

In modern systems, ALS often serves as a candidate generation stage rather than the final ranker. Its item vectors support fast nearest-neighbor retrieval, and its scores feed a later learning-to-rank model in a multi-stage ranking pipeline.

  • Log-scale or cap raw counts before computing confidence so a few heavy users do not dominate.
  • Evaluate on a time-based split, because random splits leak future behavior into training.
  • Fold in new users by solving one least squares step against fixed item factors instead of retraining.
  • Retrain on a schedule, commonly daily, since ALS is batch-oriented.

Limitations and common pitfalls

ALS inherits the limits of pure collaborative filtering. It cannot score items or users without interactions, so the cold start problem requires content features or hybrid models. It also tends to amplify popularity bias, because frequently consumed items receive strong, well-estimated vectors.

Common mistakes include treating implicit data as explicit ratings, tuning only for RMSE when the product shows ranked lists, and setting alpha so high that recommendations collapse toward bestsellers. Another frequent error is forgetting to regularize, which lets users with few interactions receive extreme vectors.

A plain-language summary

Alternating least squares is a way for a recommendation engine to learn hidden tastes from what people have watched, bought or rated. It describes every person and every product with a short list of numbers, so that people and products that fit together end up with similar numbers.

The clever part is that it adjusts one side at a time. First it keeps the product descriptions fixed and improves the descriptions of people, then it does the reverse. Each round gets a little better, much like tuning two dials by turning one, then the other, until the sound is right.

An example from practice

A mid-sized online music service has about two million listeners and 500,000 tracks, but each listener has played only a few hundred tracks. The team builds a play-count matrix, applies a log transform, and trains implicit ALS with rank 128, lambda 0.05 and alpha 20 on a small cluster in under an hour each night.

The resulting track vectors feed an approximate nearest-neighbor index that proposes 500 candidates per listener, which a gradient-boosted ranker then orders using recency and context. Compared with a popularity baseline in an online test, the ALS-based pipeline raises recall at 50 substantially and increases the share of plays from personalized suggestions, while new listeners are served by a content-based fallback until they have enough history.