Deep learning recommendation model

Also known as: DLRM

On this page 7

The deep learning recommendation model (DLRM) is a neural network architecture for click-through rate prediction that combines embedding tables for categorical features, multilayer perceptrons for numerical features and an explicit pairwise feature interaction layer. In recommender systems, it serves as a reference design for the ranking stage, where a model scores candidate items for one user and request.

Researchers at Meta published DLRM in 2019 together with an open-source PyTorch implementation. It became widely known because it exposed the systems problem behind industrial recommendation: the model is small in compute but enormous in memory, since its embedding tables can reach hundreds of gigabytes or more.

How the DLRM architecture processes features

DLRM splits its input into two kinds of features and handles each with a dedicated component. Dense features, such as a user’s age or the number of past visits, are continuous values. Sparse features, such as a user ID, an item ID or a device type, are categorical values drawn from very large vocabularies.

ComponentInputRole
Bottom MLPDense numerical featuresProjects them to the embedding dimension
Embedding tablesSparse categorical IDsMap each ID to a learned vector
Interaction layerAll vectors from both pathsComputes pairwise dot products
Top MLPInteractions plus dense vectorLearns higher-order combinations
Sigmoid outputFinal top MLP valueReturns a click probability
Main components of a DLRM

Each sparse feature has its own embedding table. When a feature contains several IDs at once, for example the last 20 items a user viewed, the model looks up all of them and pools the vectors, usually by summation. The bottom MLP turns the dense features into a vector of the same length, so that all vectors can interact.

The interaction layer computes the dot product of every pair of vectors, following the logic of factorization machines, and concatenates these scores with the dense vector. The top MLP then learns nonlinear patterns from them, and a sigmoid produces the predicted probability that the user clicks or converts.

Why embedding tables dominate DLRM systems

DLRM is memory-bound rather than compute-bound. Its MLPs contain a few million parameters, while the embedding tables can hold billions of rows in production, which places very different demands on hardware than vision or language models.

Training therefore uses hybrid parallelism. The embedding tables are split across devices with model parallelism, while the MLPs are replicated with data parallelism. Between the two parts, an all-to-all exchange sends looked-up vectors to the devices that process each training example, and this communication step often limits throughput.

TechniqueHow it worksTrade-off
Hashing trickMaps many IDs into fewer rowsCollisions blur rare items
Quotient-remainder trickCombines two smaller tables per featureSmall accuracy cost
Mixed-dimension embeddingsShorter vectors for rare IDsExtra tuning effort
Tensor-train decompositionFactorizes a table into small coresMore compute per lookup
Low-precision storageKeeps rows in 8 or 4 bitsSlight quality loss
Common techniques for shrinking embedding tables

Inference faces the same pressure. Serving systems often keep the most frequently accessed rows in GPU memory and the long tail in host memory or on fast SSDs, because access patterns follow a heavy-tailed popularity distribution.

DLRM compared with other ranking models

DLRM belongs to a family of deep CTR prediction models that differ mainly in how they model feature interactions. Its design choice is deliberate simplicity: it limits explicit interactions to second-order dot products and leaves anything more complex to the top MLP.

ModelInteraction approachKey difference from DLRM
Factorization machinePairwise dot products, no deep layersNo MLPs for nonlinear patterns
Wide & DeepLinear wide part plus deep networkManual cross features in the wide part
DeepFMFM component shares embeddings with a DNNFM and DNN run in parallel
Deep & Cross NetworkLearned cross layers of bounded degreeExplicit higher-order crosses
Two-tower modelSeparate user and item encodersUsed for retrieval, not fine ranking
How DLRM relates to neighboring architectures

Later versions of the MLPerf benchmark replaced the original dot-product layer with DCN-v2 cross layers, reflecting a broader trend toward richer interaction modules. The overall pattern of embeddings, interactions and MLPs remains the standard template for production ranking models.

Where DLRM fits in a recommendation pipeline

A DLRM is typically used in the ranking stage of multi-stage ranking. Candidate generation, often a two-tower model with approximate nearest neighbor search, first narrows millions of items to a few hundred or thousand. The DLRM then scores each candidate precisely, using features that would be too expensive to apply to the whole catalog.

  • Training data comes from logged impressions with click or conversion labels, a form of implicit feedback.
  • Evaluation relies on offline metrics such as AUC and normalized cross-entropy, followed by online A/B tests.
  • Freshness matters: many teams retrain or fine-tune daily because user interests and the catalog change quickly.
  • Benchmarks such as MLPerf use DLRM on the Criteo click logs to compare training and inference hardware.

Limitations and common pitfalls of DLRM

DLRM is a strong baseline, but it inherits the weaknesses of models trained on logged behavior. Most failures stem from data and feedback effects rather than from the architecture itself.

  • Cold start: new users and items have untrained embedding rows and receive poor scores.
  • Position bias: clicks on top slots inflate labels unless position is modeled or corrected.
  • Feedback loops: the model learns from items it already promoted, which can narrow diversity.
  • Overfitting of embeddings: rare IDs memorize noise, so frequency thresholds and regularization help.
  • Sequence blindness: sum pooling ignores order, so sequential models capture recent intent better.

Teams also underestimate the infrastructure cost of a DLRM. Sharding tables, balancing all-to-all traffic and keeping embedding caches warm usually require more engineering than the model code.

Stripped to the essentials

A deep learning recommendation model guesses how likely a person is to click on something, such as a video, a product or an advertisement. It turns every user, item and category into a short list of learned numbers, then checks how well these lists match each other.

Think of it as a matchmaker with a huge card index. Each card describes one person or one item, and the matchmaker compares cards in pairs before making a final judgment. The card index is enormous, which is why running these models takes large amounts of memory rather than raw computing power.

Where it is applied

An online marketplace with about 40 million products must decide which 20 items to show on each shopper’s home page. A two-tower model first retrieves around 1,000 candidates in a few milliseconds, and a DLRM then ranks them using features such as the shopper’s recent categories, the device, the time of day and each product’s price band.

The largest embedding tables, for product IDs and shopper IDs, are sharded across eight GPUs, while frequently viewed products stay cached for serving. The team retrains the model every night on the previous day’s impressions. After replacing an older gradient-boosted ranker, an A/B test shows a measurable rise in click-through rate and add-to-cart events, and the team adds a diversity rule to keep the feed from repeating the same categories.