The second half of the round: twists and deep dives

The first half of a design round shows that you can build a simple design. The second half shows how you think when the problem moves. The interviewer says: "Now there are 10 times more users." Or: "A server just stopped." Or: "The product team wants a new feature." Nobody can prepare for the exact change, so the interviewer sees how you work.

A weak answer panics, draws the whole design again, or defends the old design. It waits to be told which part to examine. A strong answer stays calm. It says the new number, finds the first part that breaks, changes only that part, and says what the change costs. It also chooses its own deep dives and watches the clock.

At SDE-1 level you should handle a change with some help. At SDE-2 level you should do the steps alone, with numbers. At SDE-3 level you should choose the hardest part yourself and say when you would change the design again.

Go back to the cook in lesson 2, who plans food for a family function. Halfway through the cooking, the host says: "Twenty more guests are coming." A good cook does not throw away the food and start again. The cook counts: is there enough rice? Enough plates? Enough helpers? The cook finds the first thing that is short, fixes that, and checks the next one.

The same works in a design round. Lesson 2 gave the clock for 45 minutes. The first five steps take 24 minutes (5 + 2 + 4 + 3 + 10). Deep dives take 18 minutes, from minute 24 to minute 42. The closing takes the last 3 minutes. This lesson is about those 18 minutes.

The twist

A twist is a sudden change that the interviewer adds after your first design. It has three kinds:

1. The numbers change. There are 10 times more users, or the page must open in half the time.
2. A part breaks. A server stops, or the database becomes slow.
3. A need changes. There is a new feature, a smaller budget, or users in another country.

A twist is not a sign that your design was bad. A good design needs a change in only one or two places.

Five moves for a changed requirement

1. Say it back. Repeat the change in one sentence, with the number. "So traffic goes from 667 to about 6,667 requests per second. Is that right?" This shows that you listened, and it gives the interviewer a chance to correct you.
2. Ask one question. Ask about the thing that decides the answer. "Do the results change after they are published?" If the interviewer says "assume what you like", say your assumption aloud (lesson 6).
3. Redo the arithmetic for what changed. Say each step.
4. Find what breaks first. For each part, work out load divided by limit. A result above 100% means the part cannot cope. List the parts above 100%, starting with the one furthest over. If adding more copies fixes a part (more servers, as in lesson 2), say so in one sentence. If adding copies does not fix it (one database), the design must change. Spend your time there, and keep the rest.
5. Say the trade-off. Use the sentence from lesson 6: it solves, it costs, I would change it when. Then update your requirements and your drawing, and say that you did.

Do not restart. Do not defend the old design. Do not add parts that have no number behind them.

Choosing two deep dives

Sometimes the interviewer names the part. If not, you choose. Rank the parts with three tests:

1. Nearest its limit. The highest load divided by limit.
2. Worst if it fails. A single point of failure is one part that stops the whole system when it breaks. Look for it.
3. What the problem is about. The part that makes this problem hard.

Take one dive about growth and one about failure, so that the two dives show different ideas. A twist that the interviewer gives counts as one dive.

One deep dive in five beats

1. Name the part and say why: "I will look at the database, because it is nearest its limit."
2. Give the number: load divided by limit.
3. Give two options and compare them (lesson 6).
4. Choose, with the trade-off sentence.
5. Check back: "Does this answer your question, or should I look at another part?"

The clock for the 18 minutes

Say the time aloud once: "I have about 7 minutes for this part." If one dive runs 2 minutes over, shorten the next one. Never skip the closing.

If you are stuck, say what you know and make an assumption. If the interviewer redirects you, follow at once. It is a hint, not an attack. If you make a mistake, say "I made a mistake, let me fix it" (lesson 6).

We continue the results website of lesson 2. The design has the part that shares requests, then 3 servers, then 1 database. We are at minute 24, and the deep dives begin.

Assumptions (from lesson 2, and new ones made up to teach with):

• Before the twist: 2 lakh (200,000) students, 2 opens each, in 10 minutes. That is 400,000 / 600 = 667 requests per second.
• A server answers 500 requests per second. The database answers 2,000 lookups per second.
• A cache is a fast store in memory that keeps copies of data that is read often (lesson 31). One cache server answers 50,000 reads per second. One result takes 1 KB.

Minute 24: the twist. The interviewer says: "Next year 20 lakh (2,000,000) students will check in the same 10 minutes. The page must still open in 2 seconds."

Move 1: say it back. 2,000,000 x 2 = 4,000,000 requests. 4,000,000 / 600 = 6,667 requests per second. That is 10 times more.

Move 2: ask one question. "Do the results change after they are published?" The interviewer answers: "No. The board loads the marks before results day, and they stay fixed."

Move 3: arithmetic. Servers: 6,667 / 500 = 13.3, so 14. If one may break, 14 servers are not enough: after one breaks, 13 x 500 = 6,500 is below 6,667. So we need 15: after one breaks, 14 x 500 = 7,000 is enough, but only just (95% busy).

Move 4: what breaks first. Servers: 6,667 / (3 x 500) = 6,667 / 1,500 = 4.4 times over. Database: 6,667 / 2,000 = 3.3 times over. This answers the predict box: the servers are further over, by 4.4 times against 3.3 times. We fix the servers by adding more, as lesson 2 said. We cannot fix the database by adding servers, so it needs a change in the design. That is where we spend the time.

One idea is a cache. Suppose we fill it as students ask. Each student opens twice, and the first open of each result is a miss. So at best half of the reads hit the cache. The database still gets 6,667 / 2 = 3,333 lookups per second, which is 167% of its limit. Instead, preload all the results into the cache before the opening, because they do not change.

• Memory: 2,000,000 x 1 KB = 2 GB.
• Load time: 2,000,000 lookups at 2,000 per second = 1,000 seconds, about 17 minutes. We do it the night before.
• At the peak, all reads hit the cache. The database sees close to 0 lookups per second. One cache server needs 6,667 / 50,000 = 13% of its speed.

Move 5: the trade-off. We say: "Preloading the results into a cache solves the database limit. Lookups fall from 6,667 per second to close to 0. It costs 2 GB of memory and a 17-minute load before the opening. I would change it if marks could change after publishing, because then the cache could show old marks." We also update the requirements: 6,667 requests per second, 15 servers, one cache.

Minute 31: choose my own dive. I list the parts with their load divided by limit, and what happens if each fails.

The cache is a single point of failure. That is dive 2, about failure.

Beats 1 and 2: "I will look at the cache, because if it fails at the busiest minute, the site goes down. The database would be 333% over its limit." Beat 3, two options:

• A: two cache servers, each holding all results. If one fails, the other serves everything. Cost: another 2 GB and another server.
• B: no second cache. If the cache fails, the database answers 2,000 of 6,667 requests, which is 2,000 / 6,667 = 30%. The other 70% get a fast error. Cost: nothing extra, but 70% of students see an error.

Beat 4: "Two cache servers solve the single point of failure. They cost 2 GB more and one more server. I would use option B if the board could accept errors for a few minutes." Beat 5: "Does this answer your question?"

That took minutes 31 to 38. Minutes 38 to 42 are for the interviewer's questions. At minute 42 the closing starts: "The site needs 6,667 requests per second. Fifteen servers, a preloaded cache on two servers and one database meet it, even when one server or one cache breaks."

• Keeping the design and fixing the first break solves wasted minutes and a confused interviewer. It costs a design that a fresh start might beat. I would redraw only if the fix changes most of the parts. This is our own guide, not a rule.
• Asking one question solves a wrong guess (here, "do results change?" decided the cache idea). It costs about 30 seconds. I would skip it when the interviewer says "assume".
• Choosing your own dives solves waiting for hints. It costs the risk of picking a part that the interviewer does not care about. Beat 5 reduces that risk.
• Two deep dives with numbers solve the lack of depth that five shallow ones leave. They cost the parts you do not examine. Say which parts you skipped.

"A twist means my first design was wrong." A twist tests how you adapt. In the example, the design kept its shape. We added servers and a cache.

"I should redraw everything so the design is clean." A new drawing would take about 10 minutes in lesson 2's clock. Fix the part that breaks, keep the rest, and say what changed.

"A deep dive should cover as many parts as possible." A dive without a number shows nothing. Two dives with load divided by limit show how you think.

1. Name the five moves for a changed requirement, in order. What are the three tests for choosing a deep dive?
2. The results site gets 4 times more students than before: 8 lakh (800,000), 2 opens each, in 10 minutes. (a) How many requests per second? (b) Load divided by limit for the 3 servers (500 each) and for the database (2,000)? (c) How many servers do you need if one may break?
3. The interviewer's twist takes you 12 of your 18 minutes. What do you do with the rest?