What is system design?

In a coding interview, there is usually one correct output. A system design interview is different. The interviewer gives you a short, open problem, such as "build a website where students check their exam results". Then they watch how you decide what to build.

There is no list of right answers. Two strong candidates can draw two different designs and both pass. What the interviewer judges is how you choose. Did you ask what the system needs? Did you pick parts that fit those needs? Did you say what each choice costs?

A weak answer draws a big diagram from memory, full of famous tool names, and never asks a question. A strong answer starts with "How many students, and how fast must the page open?" Then it picks the simplest design that works, and says what it gives up.

Interviews use level names. SDE means software development engineer. SDE-1 is an early-career level, SDE-2 is the middle level, and SDE-3 is a senior level. These level notes are our own guide, and companies differ.

At SDE-1 level you should explain your choices clearly. At SDE-2 and SDE-3 level you should also compare two options and say when you would switch.

Think about building a house. Every family wants a good house, but there is no single "correct" house. A family of two in a small town needs one kind. A family of ten in a busy city needs another. A bigger house costs more money. A house near the office costs more rent. Every choice gives you something and costs you something. A good builder first asks what the family needs. Then the builder picks what fits.

Software works the same way. A system is a set of parts that work together to do a job for users. Here are the most common parts:

• A client is the app or web browser that the user holds, on a phone or laptop.
• A request is a message from the client asking for something, for example "show me the result for roll number 4821".
• A server is a program that waits for requests and sends back answers. The word also means the computer it runs on.
• A database is a program that stores data safely, so it is still there after a restart.

System design means deciding which parts a system has, how they talk to each other, and where the data lives. The goal is a system that does what its users need. You make these choices before you write most of the code, because they are hard to change once the code exists.

The smallest system is one server on your own laptop. This one command starts a tiny web server:

python -m http.server 8000

It serves the files in the current folder. Open http://localhost:8000 in a browser on the same laptop to see them. (localhost means "this same computer".) For one person, that is a complete system. It is not a good system for 20 lakh (2,000,000) students. The job is the same, but the needs are different. (Python's own guide also says this small server is not meant for real users.)

The needs of a system come in two kinds:

1. What it must do, for example "a student types a roll number and sees the marks".
2. How well it must do it. Give a number for each: how many users, how fast the page must open, and how bad an hour of failure is.

A trade-off is a choice where you gain one thing and lose another. Almost every choice in system design is a trade-off. These are the things you usually trade:

• Speed: how fast a user gets the answer.
• Cost: how much money the servers and the people who run them cost.
• Simplicity: how easy the system is to build, understand and change.
• Staying up: whether the system still works when one part breaks.
• Correct data: whether every user always sees the latest, right data.

You cannot have the best of all five at once. More servers help you stay up, but cost more and are more work to run. A simpler design is cheaper and easier to change, but it may stop working on a busy day.

So a design is not "right" or "wrong" on its own. It is right for some needs. That is why the interviewer has no list of right answers. They judge whether your choices fit the needs, and whether you can say what each choice costs.

Two groups want a website where students type a roll number and see their marks on results day.

Assumptions (made up to teach with, not real traffic):

• One server can answer 500 requests per second.
• Each student opens the result page 3 times (they check, refresh, and show a friend).
• Each server costs the same amount per month. Call that amount 1 unit of cost.
• 1 lakh = 100,000, so 20 lakh = 2,000,000.

Group 1: a small college.

• Assumption: 2,000 students, and all of them check in the first 10 minutes.
• Requests = 2,000 students x 3 = 6,000 requests.
• 10 minutes = 10 x 60 = 600 seconds.
• Requests per second = 6,000 / 600 = 10.
• One server can do 500, so it is 10 / 500 = 2% busy.

If this one server breaks for an hour, students wait. The college can also put the marks on the notice board. That is an annoying failure, not a disaster.

Group 2: a national exam board.

• Assumption: 20 lakh (2,000,000) students check in the first hour.
• Requests = 2,000,000 x 3 = 6,000,000 requests in that hour.
• Assumption: half of them come in the first 10 minutes, because everyone checks at once.
• Requests in the first 10 minutes = 6,000,000 / 2 = 3,000,000.
• Requests per second = 3,000,000 / 600 = 5,000.
• Servers needed = 5,000 / 500 = 10 servers.

Now ask: what if one server breaks at the busiest moment? Then 9 servers are left, and 9 x 500 = 4,500 requests per second. That is less than 5,000, so some students get errors. With 11 servers, one can break and 10 x 500 = 5,000 are still left. So the board needs 11 servers to keep working when one fails.

Now test each design on the other group.

• Design A (1 server) at the board: 5,000 / 500 = 10. The board sends 10 times more requests than one server can answer. The site fails for most students on the most important day. Design A is wrong for the board.
• Design B (11 servers) at the college: the servers can answer 11 x 500 = 5,500 requests per second. The college sends 10. So the servers are 10 / 5,500 = 0.18% busy. The college pays 11 units instead of 1, and runs 11 servers instead of 1. Design B is wrong for the college.

Here is the answer to the predict box: it depends on the needs. Here the need that decides is how many requests arrive per second, and how bad it is if the site stops. Neither design is better on its own. Each one is right for one group and wrong for the other.

This is what an interviewer wants to see. Ask for the numbers first, then choose, then say the cost.

Here are the two designs, each said in three short sentences.

• Design A (one server). It solves the college's need at the lowest cost, with the fewest parts to run. It costs staying up: if the server breaks, nobody sees results until it is fixed. I would change it when requests come close to what one server can answer, or when an hour of failure is no longer fine.
• Design B (eleven servers). It solves the board's need and keeps working when one server breaks at the busiest moment. It costs 11 times more, and there are 11 servers to look after. I would change it when traffic falls far below 5,000 per second, because then the extra servers are wasted money.

A useful habit: for every choice, say what it solves, what it costs, and when you would change it. The interviewer cannot read your mind. If you do not say the cost, they assume you did not see it.

Start simple. Add parts only when a number shows you need them. Every extra part can break, and someone must look after it.

"There is one correct design, and the interviewer is waiting for it." There is not. The interviewer judges your reasons. In the example, one server was right for the college and wrong for the board. A design is right only for a set of needs.

"More servers and more famous tools make a better design." They make a more expensive and more complex design. Eleven servers for 10 requests per second wastes 10 units of cost and adds parts that can break. The interviewer will ask why each part is needed, and "it looks strong" is not a reason.

"If I say what my design costs, the interviewer will think it is weak." It is the opposite. Every design has a cost. Saying it shows that you understand your choice. Hiding it, or not seeing it, is what looks weak.

1. What is a trade-off? Name two things a designer often has to trade against each other.
2. A coding-contest website expects 30,000 students. Each opens the problem page 2 times in the first 5 minutes. One server answers 500 requests per second. (a) How many requests per second arrive? (b) How many servers do you need if the site must still work when one server breaks?
3. A candidate is asked to design a lunch-ordering app for one office of 50 people. They draw 20 servers and five different tools. What is wrong, and what should they have asked first?