Case study: Feed ranking end to end

The interviewer says: "Design the home feed of a social app with 200 million daily users. Show posts each user will like, not only the newest. It must open in under 300 ms. What happens when the ranker is down? How do you ship a new ranker?"

A weak answer ranks every post with one big model, and forgets time, cost, logging and failure. A strong answer builds a funnel with numbers for each stage. It says where features come from and what is logged. It tests a new ranker in steps, and it names the fallback.

At SDE-2 level you should draw the funnel and give a time budget. At SDE-3 level you should add the cost, the experiments and the failure plan.

A newspaper has an editor. Reporters bring stories: from people you follow, and from others. An assistant quickly picks 100 from 500. The editor orders the best 50 of them, and 20 go on the page. A desk checks the rules. After printing, the paper records what readers read, and next month's choices improve. If the editor is ill, the assistant prints the stories newest first.

Lesson 110 built a feed with 300 candidates, a ranking hook and a simple score. It saved the first 100 ordered IDs for 10 minutes. This lesson puts the funnel of lessons 167 and 168 behind that hook.

Requirements

• A user posts, follows, and opens a feed of 20 posts a page. The feed mixes posts of followed accounts with recommended posts, and has ads.
• p99 under 300 ms for a feed open. The feed must always open, even if the model fails.
• A new post can be recommended within a minute. The model is retrained daily. Every shown post is logged.

The path of one feed open

1. Candidates, in parallel. The timeline of lessons 108 and 110 gives 300 post IDs. A nearest-neighbour search (lesson 168) over the vectors of posts from the last 48 hours, using the user vector, gives 200 recommended posts. The total is 500. A new post gets its vector when it is created and joins the index, so it can be a candidate within a minute.
2. Features, in one batch call to the online store (lesson 164): one user record and 500 post records. A server keeps post records in memory for a few seconds, because many users share the same post.
3. Light ranker: a small model on ordinary processors scores 500 candidates and keeps the best 100.
4. Heavy ranker: a larger model on accelerators (lesson 166) scores the best 50 of those and orders them.
5. Mixing and rules: at most 2 posts from one author in the first 10, blocked and muted accounts removed, safety rules, and ads in set slots.
6. Save and answer: save 100 IDs for 10 minutes (the 50 from the heavy ranker, then the other 50 in light-ranker order). Load 20 post bodies. Page 2 reads the saved list.
7. Log each impression, a post shown to a user (the app reports it when the post appears on screen), to a queue, not in the request path.

A reopen inside 10 minutes reads the saved list and skips steps 1 to 5. This reuse is a rule added here: lesson 110 saved the list only to keep paging steady. The cost is that a reopen shows no new posts for up to 10 minutes.

The loop around it

Impressions and actions (like, comment, share) are joined into rows with labels (lesson 163). A daily job trains the two rankers and writes versioned model files. A new version goes through shadow mode, a canary and an A/B test (lesson 165) before it is live. Monitoring watches inputs, outcomes and model age.

When the ranker fails

The features, the light ranker and the heavy ranker run under one timeout. If it fires, use the saved list when there is one. If not, use the timeline in newest-first order, which lesson 110 already can do. If only the heavy ranker fails, keep the order of the light ranker. A circuit breaker (lesson 83) skips the ranker for a short time after many timeouts.

Assumptions (made up to teach with). Lesson 110 gives the scale: 200 million daily users, 10 opens a day each, peak 3 times the average.

• 40% of opens fall inside the 10-minute saved list (an assumption). Feature records: user 1 KB, post 100 bytes; a server keeps 90% of post records in memory. One store node does 100,000 key reads a second.
• Light ranker: 0.02 ms per candidate on one core. Heavy ranker: a batch takes 6 ms plus 0.1 ms per post (lesson 166). A server has 4 cores at Rs 12 an hour, and is at most 70% busy. An accelerator is Rs 120 an hour. A store or index node is Rs 30 an hour.

Step 1: traffic. Opens: 200,000,000 x 10 = 2,000,000,000 a day = 23,148 a second, peak 69,444 (lesson 110). Ranked opens: 69,444 x 0.6 = 41,666.4 a second at peak. We round this to 41,666 and use it from here. A day has 2,000,000,000 x 0.6 = 1,200,000,000 ranked opens.

Step 2: time budget (p99).

The sum is 120 ms of 300 ms. Lesson 110 gave a simple formula a timeout of 50 ms. This stage normally takes 20 + 10 + 15 = 45 ms (a sum of p99 times), so a 50 ms timeout would fire often. Give it 100 ms. If it fires, the open takes 10 + 30 + 100 + 5 + 20 + 10 = 175 ms. With the breaker open, it takes 75 ms.

Step 3: feature lookups. Each ranked open reads 1 user key and 50 post keys (the other 450 come from server memory): 51 keys. At the peak: 41,666 x 51 = 2,124,966 keys a second. Assumption: a store node does 100,000 key reads a second. So 2,124,966 / 100,000 = 21.2, which is 22 nodes, and 44 with two copies. Without the memory, 41,666 x 501 = 20,874,666 keys would need 209 nodes. The vectors of the last 48 hours are 80,000,000 x 512 bytes (128 numbers) = 41 GB. Exact search over them is 80,000,000 x 128 = 10.24 billion steps, 1.02 seconds, so use approximate search (lesson 148). Assumption: an index node holds all the vectors and answers 10,000 approximate searches a second, at most 70% busy. Then 41,666 / 7,000 = 6 nodes.

Step 4: cost. Light ranker: 41,666 x 10 ms = 416,660 ms of work a second, which is 417 cores. At 70% busy that is 595 cores, so 149 servers: 149 x Rs 12 x 24 = Rs 42,912 a day. Heavy ranker: one request alone takes 11 ms, so 1,000 / 11 = 90.9 a second, 63.6 at 70%. At the peak, 41,666 / 63.6 = 655 accelerators: 655 x Rs 120 x 24 = Rs 1,886,400 a day. It makes 1,200,000,000 x 50 = 60 billion predictions. Add the store (44 x Rs 30 x 24 = Rs 31,680), six index nodes (6 x Rs 30 x 24 = Rs 4,320) and training (Step 6, Rs 7,200). The total is Rs 1,972,512 a day. Opens a day are 2,000,000 thousand, so that is Rs 0.986 per 1,000 opens. The heavy ranker is 96% of it. The bill counts only these ML parts, not app servers, ads or the post store. For comparison, lesson 110's formula needed 8 servers of 8 cores, which is 16 of 4 cores: Rs 4,608 a day, about Rs 0.0023 per 1,000 opens.

Batches of 3 take at most 15 + 4 + 15 = 34 ms in the worst case. The stage then takes 20 + 10 + 34 = 64 ms, inside the 100 ms timeout. Does the ranking earn this? Assumption: ads bring Rs 40 per 1,000 opens, and ranking raises that by 2%, which is Rs 0.80. We keep the 2% when the heavy ranker scores only 30 posts, which is a guess. At Rs 0.986 the ranking costs more than it earns. At Rs 0.472 it gains Rs 0.33. Every input is a made-up number: with a 1% gain, even Rs 0.472 loses. The method is the lesson, not the sign: put cost and gain per 1,000 opens side by side (lesson 161). Only ad income is counted here.

Step 5: candidates double to 1,000. Take 500 from the timeline (the feed list of lesson 108 holds 500 IDs) and 500 recommended. Recommendations (10 ms + 0.1 ms each) go from 30 to 60 ms. Features go from 20 to 30 ms, and the light ranker from 10 to 20 ms. The heavy ranker still scores 50. The total goes from 120 ms to 10 + 60 + 30 + 20 + 15 + 5 + 20 + 10 = 170 ms. At the peak, light-ranker cores go from 417 to 833 (Rs 42,912 to Rs 85,824 a day). Store keys go from 2.1 to 4.2 million a second. Do it only if candidate recall (lesson 168) rises enough.

Step 6: data and training. Impressions: 2,000,000,000 opens x 20 = 40 billion a day. A compact record of 40 bytes is 1.6 TB a day. For a 5% sample of users, the record also holds the features that serving used (80 x 4 bytes = 320 bytes). That is 360 bytes in all, so 2 billion x 360 bytes = 720 GB a day. Training reads these sampled rows. Thirty days: 60 billion rows, 21.6 TB. Assumption: 100 machines for 6 hours at Rs 12 is Rs 7,200 a day, 0.4% of the daily bill. A like, comment or share within 1 hour is label 1. The model is 7 to 31 hours old, 19 on average (lesson 163).

Step 7: experiments. Shadow mode first: the new heavy ranker scores the same 50, and nothing changes on screen. Compare p99 and the spread of scores. Then a 1% canary for the guardrails. Then an A/B test. The online metric is the share of users with at least one like, comment or share in a day, assumed 60%. To see 0.5 points: 16 x 0.6 x 0.4 / (0.005 x 0.005) = 153,600 users for each group. A 1% group has 2,000,000, so time is the limit: run two whole weeks for the novelty effect (lesson 165). Guardrails: p99, hide and report rate, and return the next week. A model with +3% likes and +20% hide rate is not shipped. After release, watch for drift (lesson 165). Alert when missing features pass 2%, or when the mean predicted like chance and the real one differ by more than 10% (both limits are assumptions).

Step 8: failures (the predict box).

The user who opened 3 minutes ago sees the saved list. The user who opened 2 hours ago sees the newest-first timeline. Assumption: interactions per 1,000 opens are 100 ranked, 97 with the light order, 92 newest first, 0 for an error page. One hour of peak has 41,666 x 3,600 = 150 million ranked opens. Newest first loses 150,000 x 8 = 1.2 million interactions. An error page loses 15 million.

Step 9: the interview walk-through (about 35 minutes).

1. Requirements and the 300 ms target (3 minutes).
2. Numbers: 69,444 opens a second, 41,666 ranked (4 minutes).
3. The funnel: candidates, features, light, heavy, rules, save, log (6 minutes).
4. Deep dive 1: the time budget, lookups and cost with levers (7 minutes).
5. Deep dive 2: logging what was shown, daily training, no skew (6 minutes).
6. Experiments and the fallback chain (6 minutes).
7. What I would change at 10 times the size (3 minutes).

Real-world note (Instagram, 9 Aug 2023). Meta described Explore in four stages: retrieval, a light first ranker, a heavy second ranker on the best 100, and a final re-rank for rules and variety. It also precomputes recommendations for some users off-peak. Source: https://engineering.fb.com/2023/08/09/ml-applications/scaling-instagram-explore-recommendations-system/ (opens in a new tab).

Real-world note (X, README read 26 Oct 2026). The open-source repository lists candidate sources, a light ranker, a heavy neural ranker, and a mixing and filtering step. Source: https://github.com/twitter/the-algorithm (opens in a new tab).

The funnel solves a relevant feed within 300 ms. Scoring all 500 candidates with the heavy model would take 6 + 0.1 x 500 = 56 ms a request. That needs 3,334 accelerators, 5 times more than 655. It costs a recall gap, two models and Rs 0.986 per 1,000 opens. I would make the ranker smaller when its gain is below its cost.

The fallback chain solves a feed that must always open: 92 interactions per 1,000 opens, not 0. It costs a simple path that must stay tested. I would test it with a monthly switch-off.

Daily training solves a model that follows the world. It costs a model up to 31 hours old. I would train more often when posts change taste within hours.

"A better model is always worth serving." The ranking cost Rs 0.986 per 1,000 opens against an assumed gain of Rs 0.80. Batching cut the cost to Rs 0.472.

"If the ranker is down, the feed is down." The saved list or the newest-first order keeps it open. Lesson 110 had this path before any model existed.

"Log the posts that users liked." Log what was shown, with position and features. The likes alone hide the posts that nobody saw.

1. Name the stages of the funnel in order. What is logged for each impression, and why?
2. An app has 10,000,000 daily users, 8 opens each, peak 3 times the average, and 50% of opens use the saved list. Each ranked open has 400 candidates, and servers keep 90% of post records in memory. (a) How many ranked opens a second at peak? (b) How many store keys a second (1 user key plus the other post keys)? (c) The light ranker needs 0.02 ms per candidate. How many cores of work?
3. An A/B test shows +3% likes and +20% hide and report rate. What do you do?