From users to requests per second

After the requirements step (lesson 3), many interviewers say: "Roughly how many requests per second will this get?" They are not asking for the exact answer. They want to see simple steps from the number of users to the load on the system, with numbers that you say aloud.

A weak answer is "millions of requests", or a long sum with many digits that nobody can follow. A strong answer starts from the daily users, says each assumption, divides by the seconds in a day, and then thinks about the busiest time. It also separates requests that only look at data from requests that change data.

At SDE-1 level you should do the sum when the numbers are given. At SDE-2 level you should choose the assumptions and the busiest-time number yourself. At SDE-3 level you should also check that the answer is reasonable, and say which part of the system it puts under most load.

Think about a tea stall. The owner knows that 600 customers come in a day. The stall is open 12 hours, so the owner expects 600 / 12 = 50 customers an hour. But customers do not come evenly. Most come in the morning and at 5 pm. So the owner plans for the rush hour, not for the average hour.

Load on a system works the same way. You go in four steps.

Step 1: how many people. Daily active users (DAU) is the number of different people who use the system on one day. It is not the number of people who ever signed up. Monthly active users (MAU) is the same for one month. If you are given total users or MAU, ask what share is active on a day, or state that share as an assumption.

Step 2: how many requests per person. Walk through one day of one user: "opens the app 5 times, and each time the screen needs 8 requests." Then:

• requests per day = DAU x requests per user per day

Step 3: requests per second. A day has 24 x 60 x 60 = 86,400 seconds. So:

• average requests per second = requests per day / 86,400

For a quick answer without a calculator, you can divide by 100,000 instead. The answer is about 14% too low, which is fine for a rough number. Say that you rounded.

Step 4: the busiest time. Traffic is not even. The peak is the highest load over a short time, such as the busiest hour. The peak factor is the peak divided by the average. You can choose it as an assumption, and say it aloud. For an app with an evening rush, we suggest a first guess between 2 and 5. This is our own advice, not a standard. Or you can work it out. Suppose a share s of the day's requests arrive in the busiest w hours. Then:

• peak requests per second = (s x requests per day) / (w x 3,600)
• peak factor = s x 24 / w

Design for the peak, not the average. A user at the busiest time does not care that the average is low. A special event, such as a launch or a sale, can be many times the average. Ask if one is expected.

Reads and writes

A read is a request that only looks at data and changes nothing, such as opening a page. A write is a request that changes data, such as saving a post or a like. The read to write ratio is the number of reads divided by the number of writes. A ratio of 20:1 means 20 reads for each write.

The ratio tells you which kind of request to worry about first. An app where people mostly look at things has many more reads. An app that saves events all day has more writes. Give the ratio as an assumption, and work out reads and writes separately.

Numbers worth memorising

These numbers are rough. They change with hardware and with time, so use them to compare sizes, not as exact facts.

Time. 1 day = 86,400 seconds. 30 days = 2,592,000 seconds (about 2.6 million). 1 year = 31,536,000 seconds (about 31.5 million). From these, 1 million requests a day is 1,000,000 / 86,400 = 11.6, so about 12 per second. 100 million a day is about 1,157 per second. 1 billion a day is about 11,574 per second.

Powers of two. A bit is a 0 or a 1. A byte is 8 bits and holds one English letter. A number with n bits can hold 2 to the power of n different values. Each step of 10 in the power multiplies the value by about 1,000 (exactly 1,024).

A 32-bit number holds 2 to the power of 32 = 4,294,967,296 values, about 4.3 billion. For estimates, we use 1 KB = 1,000 bytes. In India, 1 lakh = 100,000 and 1 crore = 10,000,000 (10 million).

Sizes. These are assumptions for teaching:

• An English letter is 1 byte. Most Hindi letters take 3 bytes in the common text format (UTF-8). So a 280-letter English post is 280 bytes, and the same length in Hindi is about 840 bytes.
• A small database record is about 1 KB.
• A phone photo, after the app shrinks it for upload, is about 300 KB. A smaller version for a phone screen is about 100 KB.

Times. The table below is a widely shared list of rough times. It is not measured on one machine, and newer machines are faster. Memory is where a running program keeps data. It is fast, but it loses the data when the power goes off. A disk or an SSD (a drive with no moving parts) keeps data without power, but it is slower. A data centre is a building full of computers (lesson 13). 1 ms = 1,000 microseconds (µs) = 1,000,000 nanoseconds (ns).

Read it as a ladder: each step is much slower than the one above. A disk seek is 10 ms / 0.0001 ms = 100,000 times slower than memory. An SSD read is 0.15 / 0.0001 = 1,500 times slower than memory.

We estimate the load of a photo-sharing app.

Assumptions (made up to teach with, not real traffic):

• DAU = 1 crore = 10,000,000.
• A user opens the app 5 times a day. Each time, the screen makes 8 read requests. That is 5 x 8 = 40 reads.
• A user makes 2 write requests a day (likes, comments and new posts together).
• Half of the day's requests arrive in the busiest 4 hours.

Step 1: requests per day.

• Reads = 10,000,000 x 40 = 400,000,000
• Writes = 10,000,000 x 2 = 20,000,000
• Total = 420,000,000 (42 requests per user)

Step 2: average per second.

• Total = 420,000,000 / 86,400 = 4,861 (rounded from 4,861.1)
• Reads = 400,000,000 / 86,400 = 4,630 (rounded from 4,629.6)
• Writes = 20,000,000 / 86,400 = 231 (rounded from 231.5)

This answers the predict box: about 5,000 requests per second.

Step 3: the peak. The busiest 4 hours are 4 x 3,600 = 14,400 seconds. Half of the requests arrive then.

• Peak total = (0.5 x 420,000,000) / 14,400 = 210,000,000 / 14,400 = 14,583 (rounded from 14,583.3)
• Peak factor = 0.5 x 24 / 4 = 3. Check: 4,861.1 x 3 = 14,583.3.
• Peak reads = (0.5 x 400,000,000) / 14,400 = 13,889 (rounded from 13,888.9)
• Peak writes = (0.5 x 20,000,000) / 14,400 = 694 (rounded from 694.4)

Step 4: the ratio. Reads : writes = 400,000,000 : 20,000,000 = 20 : 1.

Step 5: a speed check with the time ladder. Assumptions: a home-screen request makes 4 reads from the database, one after another. Each read costs one disk operation. The speed target is 100 ms.

• If each read is a disk seek: 4 x 10 ms = 40 ms. That is 40% of the target.
• If each read is an SSD read: 4 x 0.15 ms = 0.6 ms. That is 0.6% of the target.

Step 6: does the answer look reasonable? The food app in lesson 3 had 1,389 requests per second at its busiest. Here 14,583 / 1,389 = 10.5 times more. The size is in the range we have seen before. If the answer were 1,000 times bigger, we would check our inputs.

In the interview you say: "About 4,900 per second on average, about 14,600 at peak, 20 reads for each write."

• It solves guessing: the design is sized from numbers that anyone can check. It costs accuracy and time. Every input is an assumption, so the answer can be wrong by 2 times or 10 times, and it takes 3 to 5 minutes (our guide). I would redo it when the interviewer gives a real number, or a number changes.
• Designing for the peak solves slow service at the busiest time. It costs more: here the peak is 3 times the average, so the system must be 3 times bigger than the average needs. I would design for less than the peak only if the business accepts slow service at the busiest time, and I would say so aloud.
• Rounding in your head solves speed. It costs a little accuracy: dividing by 100,000 gives a number about 14% low. I would use 86,400 when the interviewer wants a close answer.

"Total users is the number to use." A system with 10 crore sign-ups may have 1 crore daily users. 100,000,000 x 42 / 86,400 = 48,611 per second is 10 times too high. Use DAU, and ask for it.

"The average per second is what I design for." The average here is 4,861. The peak is 14,583. A system built for the average fails in the busiest hours, when the most users are present.

"An estimate must be exact to be useful." It must be about right. 4,861 and 5,000 lead to the same design. If an input is wrong by 10 times, exact arithmetic does not help. Spend your time on the assumptions.

1. What are DAU, the peak factor and the read to write ratio? How many seconds are in a day?
2. A notes app has 50 lakh (5,000,000) daily users. Each user sends 30 requests a day, and 1 request in 10 is a write. 60% of the day's requests arrive in the busiest 6 hours. Find (a) the average requests per second, (b) the peak requests per second, (c) the peak writes per second and the read to write ratio.
3. A candidate says: "We have 10 crore sign-ups and each user sends 42 requests a day. That is 4,200,000,000 requests a day, so 48,611 per second. I will design for that." What is wrong, and what should the candidate ask?