Recommendation systems basics: candidates, embeddings and a ranker

The interviewer says: "Design the 'try tonight' row on the home screen of a food app. There is no search box. How do you choose what to show? What does a new user see? What if the same popular dishes are shown to everyone?"

A weak answer says "use deep learning" and stops. A strong answer builds a funnel: cheap sources give a few hundred candidates, a ranker scores them, and a last step adds variety. It handles a new user and a new dish, and sees that the system learns only from what it showed.

At SDE-2 level you should explain the funnel and the candidate sources. At SDE-3 level you should also explain what a new user sees and how new items get tried. Also explain how the system learns only from what it showed, and batch against real-time.

Think of a shop assistant who knows you. You do not say what you want. The assistant remembers what you bought, what people like you bought, and what is new. Search (lesson 167) starts from a query. A recommender answers "what should this user see next?" with no query.

The funnel is the one from lesson 167. Candidate generation picks a few hundred items out of millions with cheap methods. The ranker scores them. Re-ranking applies variety and rules.

Candidate sources

Each source is cheap and gives about 100 candidates. The system merges them and removes repeats.

1. Popular items, for example the most ordered dishes in the user's city today. One list serves many users.
2. Items chosen together (item-to-item). Count how often two items are chosen by the same user. Compare it with chance using the lift: how often two items go together, divided by how often you would expect it if they were unrelated. A popular item goes with everything, so its lift is near 1.
3. People like you. Collaborative filtering learns from what many users did, not from item content. Item-to-item is one kind. In the user-to-user kind, two users who chose many of the same items have similar taste, so what one chose becomes a candidate for the other. The data is a table of users and items.
4. Embeddings and nearest neighbours. An embedding is a list of numbers for the meaning of something, where similar things have near lists (lesson 148). A two-tower model has two small models. The user tower turns user features into a vector. The item tower turns item features into a vector. Training pushes the vector of a user near the vectors of the items that user chose. At serving, a nearest-neighbour search finds the item vectors nearest to the user vector. It is exact, or approximate when the library is large (lesson 148).

Item vectors change slowly, so compute them in advance, when an item is added, and keep them in an index. The user vector depends on recent actions, so compute it when needed, or once a day.

Ranker and re-ranking

The ranker is a learned model as in lesson 167, with different features. They are the user, the item, their history together (has the user ordered this dish before?) and the source of the candidate. There is no text match. It predicts the chance of a click or an order. Re-ranking then applies rules in plain code: no sold-out or closed items, no dish the user ordered today, at most 2 dishes from one restaurant. This is the variety step.

Cold start

Cold start means that there is no history. A new user has no vector, so show popular items for the city and time of day, then use the first taps. A new item has no orders, so collaborative filtering cannot place it. An item tower that reads the item's content (cuisine, price, photo) gives it a vector on day 1. Also show it to a few users on purpose.

Explore or exploit

To exploit is to show what you already think wins. To explore is to show something else to learn about it. This is the bandit problem: a gambler with several machines must choose between the best known machine and a try of another. The simplest rule is epsilon-greedy: for a small share epsilon of the views, pick a good candidate at random. For the rest, show the best known. A fixed epsilon explores at the same rate for ever. Better rules, such as Thompson sampling, explore less as they learn.

This matters because of the feedback loop of lesson 165. The model chooses what users see, and users can only choose what they see. A popular item gets more views, so more orders, so a higher rank. Counts of orders then mislead; use the order rate, which is orders divided by views. A feed that only shows predicted winners goes stale and hides new items.

Batch or real-time

As in lesson 166, batch recommendations are computed in a night job and stored by user, which is cheap and old. Real-time recommendations are computed on each request, which is fresh. A mix is common: the night job stores the candidates, and real time ranks them with fresh features (the time of day, the last order).

Metrics and what to log

Candidate generation is judged by candidate recall: of the items the user later chose, the share that were in the candidate list. The ranker is judged by NDCG (lesson 167). Online, use orders per user and guardrails (lesson 165), plus coverage: how many different items were shown. Log, for each view, the user, the item, its position, its source, the model version, the features used and what the user did. Then training learns from what was shown, not only from what was chosen.

SwadKart (lessons 163 to 167) adds a "try tonight" row of 10 dishes.

Assumptions (made up to teach with):

• 2,000,000 dishes (lesson 112) and 2,000,000 users. 500,000 users open the app 3 times a day: 1,500,000 requests, peak 69.6 a second (lessons 163 and 166).
• The ranker needs 0.05 ms per dish on one core (lesson 166). A core-hour costs Rs 3.
• Each request shows 10 dishes, so 15,000,000 views a day. An average view gives an order with chance 1%, which is 150,000 orders a day.

Step 1: why a funnel. Scoring all 2,000,000 dishes takes 2,000,000 x 0.05 ms = 100,000 ms = 100 seconds for one request. Scoring 300 candidates takes 300 x 0.05 ms = 15 ms.

Step 2: items chosen together. Of 1,000 users who ordered biryani, 300 also ordered raita and 600 also ordered cola. Raita is ordered by 20% of all users, cola by 60%. Lift of raita = 0.30 / 0.20 = 1.5. Lift of cola = 0.60 / 0.60 = 1.0. Raita goes with biryani. Cola is only popular.

Step 3: vectors. Assumption: 64 numbers of 4 bytes each, so 256 bytes a dish. All dishes take 2,000,000 x 256 bytes = 512 MB. Exact search compares the user vector with every dish: 2,000,000 x 64 = 128,000,000 steps. At 10 billion steps a second (lesson 148) that is 12.8 ms. At the peak, 69.6 x 12.8 ms = 891 ms of work a second, about 0.9 core. Exact search is enough here. As a toy with 2 numbers: user (0.8, 0.2), biryani (0.9, 0.1), dosa (0.6, 0.4), pizza (0.2, 0.8). The distances are 0.14, 0.28 and 0.85, so biryani is nearest.

Step 4: merge, rank, re-rank. Take 100 popular dishes, 100 from item-to-item and 100 from nearest neighbours. Assumption: 40 repeat, so 260 are unique. The ranker needs 260 x 0.05 ms = 13 ms. Re-ranking removes a closed restaurant and keeps at most 2 dishes from one restaurant, then the top 10 are shown.

Step 5: cold start and exploring (the predict box). Without help, the new dish is not shown, so it gets no order. The system can use the dish's content for a vector, and it can explore. Assumption: 5% of views are exploration slots for 50 new dishes. That is 15,000,000 x 5% = 750,000 views, or 15,000 views for each new dish. At a 1% order rate that is 150 orders. A count of 150 varies by about its square root, 12, which is 8%. The cost: assume an exploration slot orders at 0.8%, not 1%. Orders lost = 750,000 x (0.010 - 0.008) = 1,500, which is 1,500 / 150,000 = 1% of orders.

Step 6: popularity. Dish A had 100,000 views and 1,000 orders. Dish B had 10,000 views and 100 orders. Both have an order rate of 1%, but A has 10 times the orders. Ranking by order count would keep A first. Use the rate, and log the views. A rate from few views is noisy: 1 order in 3 views is 33%. Wait for enough views.

Step 7: batch or real-time. A nightly job runs the vector search for all users. That is 2,000,000 x 12.8 ms = 25,600 seconds = 7.11 core-hours, which is 7.11 x Rs 3 = Rs 21.3 a night. Real time for 1,500,000 requests is 1,500,000 x 12.8 ms = 5.33 core-hours, which is Rs 16 of work. The work is similar. The difference is freshness and the fleet (lesson 166). Built at 2 am, a list is 18 hours old at 8 pm and does not know about the lunch order. So the night job stores the candidates and real time ranks them.

Step 8: measure. Assumption: of 1,000 orders in a test week, 720 were of dishes in the 300 candidates. Candidate recall = 720 / 1,000 = 72%. The ranker cannot help with the other 28%. Coverage: 40,000 different dishes shown in a day, 40,000 / 2,000,000 = 2% of the library.

Real-world note (YouTube, 2016). Google researchers described the YouTube recommender as two models: candidate generation, then a separate ranking model. Source: https://research.google/pubs/deep-neural-networks-for-youtube-recommendations/ (opens in a new tab).

Real-world note (Netflix, Oct 2026). Netflix wrote that one foundation model replaced several specialised models. Teams can take its embeddings, which are cheap but can be stale. Or they can build it into their own model or fine-tune it, which costs more. Source: https://netflixtechblog.com/integrating-netflixs-foundation-model-into-personalization-applications-cf176b5860eb (opens in a new tab) (blog feed, 26 Oct 2026, dated 14 Oct 2026).

Candidate generation solves a library too big to score: 15 ms for 300 dishes, not 100 seconds for 2,000,000. It costs recall (28% of the chosen dishes were missed), because a dish outside the candidates cannot be ranked. I would add a source when recall is low.

Exploration solves new items and a stale feed. It costs 1% of orders here. I would explore more when new items arrive every hour.

Batch candidates solve cost and a flat latency. They cost old answers (18 hours). I would rank a nightly list live.

"Show the most popular items, because they get the most orders." Dish A had 10 times the orders and the same rate as B. Popular items also get more views.

"Collaborative filtering solves cold start." It needs history. A new item has none, so use content and exploration.

"A recommender is one big model." Here it is sources, a ranker and a rules step. Each changes on its own.

1. What is collaborative filtering? What is cold start? What does epsilon-greedy do?
2. An app has 1,000,000 items with vectors of 32 numbers of 4 bytes. (a) How many MB do the vectors take, and how long does one exact search take at 10 billion steps a second? (b) Of 2,000 users who ordered X, 500 also ordered Y. Y is ordered by 10% of all users. What is the lift?
3. The home row shows the same 40 dishes to everyone, and orders are flat. What do you check and change?