Real-time recommendation (also called streaming recommendation) is the practice of generating or updating suggestions within milliseconds to seconds of a user action, so that a click, search, purchase or skip changes what is shown next during the same session. In recommender systems, it contrasts with batch recommendation, where lists are precomputed nightly and served unchanged until the next run.
The value of the approach comes from intent drift: what a person wants now often differs from their long-term profile. Capturing that shift requires streaming event pipelines, fresh features and low-latency ranking, which makes real-time recommendation as much an engineering discipline as a modeling one within artificial intelligence.
How real-time recommendation reacts to user events
A real-time recommender reacts to events through a closed loop of capture, feature update, retrieval and ranking. User interactions are emitted as events, processed by a stream processor, written into a low-latency feature store, and read by the serving stack the next time recommendations are requested.
Serving usually follows a multi-stage architecture. Candidate generation retrieves a few hundred to a few thousand items, often with a two-tower model and an approximate nearest neighbor index; a heavier ranking model scores those candidates with fresh context; and a re-ranking step applies business rules, diversity and deduplication.
The table below separates the layers of freshness that a system can implement, from simplest to most demanding.
| Freshness level | What updates in real time | Typical delay |
|---|---|---|
| Real-time features | Session and short-term counters fed to a fixed model | Under 1 s to a few seconds |
| Real-time candidates | Retrieval index and trending items | Seconds to minutes |
| Near-line model updates | Embeddings or model weights retrained on recent data | Minutes to hours |
| Online learning | Model parameters updated per event or mini-batch | Seconds to minutes |
| Batch recommendation | Precomputed lists only | Hours to a day |
Most production systems combine real-time features with periodically retrained models, because fully online learning is harder to monitor and more vulnerable to noisy or adversarial feedback.
Latency budgets and system components
Real-time recommendation must fit inside a strict end-to-end latency budget, commonly 50 to 200 ms at the 99th percentile for the whole request. Each stage receives a slice of that budget, and the slowest stage determines whether the page renders on time.
| Stage | Typical budget | Common technology |
|---|---|---|
| Event ingestion | Asynchronous, under 1 s to visibility | Message log such as Apache Kafka |
| Stream feature computation | Asynchronous, seconds | Apache Flink or similar stream processor |
| Feature lookup | 2 to 10 ms | In-memory key-value store |
| Candidate retrieval | 5 to 30 ms | HNSW or IVF vector index |
| Ranking | 10 to 80 ms | Gradient-boosted trees or deep ranking model |
| Re-ranking and filtering | 2 to 10 ms | Rules and diversity logic |
A central engineering risk is training-serving skew: features computed one way offline and another way in the stream produce models that score well in evaluation but behave differently in production. Shared feature definitions and point-in-time correct training data are the standard remedies.
Models suited to session-aware ranking
Models for real-time recommendation must consume recent interactions as input rather than relying only on a static user embedding. Sequential and session-based models are therefore common, alongside classic ranking models fed with fresh counters.
- Session-based models such as GRU4Rec-style recurrent networks predict the next item from the current session alone, which helps with anonymous users.
- Transformer sequence models such as SASRec and BERT4Rec encode the recent interaction sequence with self-attention.
- CTR prediction models such as DeepFM or Deep & Cross Network combine long-term features with real-time counters like clicks in the last five minutes.
- Contextual bandits balance exploration and exploitation and update quickly from reward signals.
- Item-to-item similarity lists react instantly to the last viewed item and remain a strong, cheap baseline.
A frequent design choice is to compute the user state at request time from the latest events instead of storing a precomputed user embedding, which keeps recommendations responsive without retraining.
Real-time versus batch and session-based approaches
Real-time recommendation is a property of the serving system, while session-based and sequential recommendation are families of models. The two are often combined, but a sequential model served from nightly lists is not real-time, and a real-time system can use a simple model.
| Approach | Reacts to current session | Infrastructure cost | Best fit |
|---|---|---|---|
| Batch recommendation | No | Low | Email digests, stable catalogs |
| Real-time recommendation | Yes | High | Feeds, shops, streaming media |
| Session-based recommendation | Yes, if served live | Medium | Anonymous or short visits |
| Context-aware recommendation | Partly, via time and device | Medium | Location or time-dependent offers |
The decision to go real-time should follow measured value. When item catalogs change slowly and user intent is stable, the added complexity may yield only small gains over hourly refreshes.
Pitfalls and how to mitigate them
Fast feedback amplifies both good and bad signals, so real-time systems are prone to feedback loops, popularity spikes and manipulation. The table pairs frequent symptoms with their causes and fixes.
| Symptom | Likely cause | Mitigation |
|---|---|---|
| Feed collapses onto one topic | Over-weighting the last few clicks | Blend short-term and long-term signals |
| Sudden promotion of junk items | Click fraud or shilling attack | Rate limits and anomaly detection on events |
| Offline gains vanish in A/B tests | Training-serving skew | Shared feature definitions and logging |
| Timeouts during traffic peaks | Ranking stage exceeds budget | Fallback to cached or batch lists |
| Same items repeated across pages | Missing impression tracking | Filter items already shown in the session |
Evaluation is also harder than in batch settings. Offline metrics such as recall at k or NDCG must be computed with time-ordered splits, and final decisions should rest on online A/B tests measuring engagement, conversion and long-term retention rather than immediate clicks alone.
The short version
Real-time recommendation means the suggestions on a website or app change while you are still using it. If you start looking at hiking boots, the next screen shows trails gear instead of the kitchen items you bought last month.
It works like an attentive shop assistant who notices what you pick up and adjusts what they show you next, instead of handing everyone the same printed catalog. The key point is reacting to what you want right now, not only to your history.
Where it makes a difference
A mid-sized online grocery retailer serves product suggestions from lists computed once per night. Analysts notice that shoppers who add pasta and canned tomatoes are still shown breakfast cereals, because the nightly lists reflect each customer’s long-term habits rather than the current basket.
The team adds a stream of add-to-cart and search events, computes basket features within about two seconds, and ranks a candidate set of 500 items with a gradient-boosted model under a 60 ms ranking budget. Complementary items such as parmesan and basil now appear while the shopper is still building the order.
In a four-week A/B test the real-time variant raises the share of orders containing a recommended item by several percentage points, and the batch lists are kept as a fallback when the stream lags.