Storage, bandwidth and how many servers

After you know the requests per second (lesson 14), the next questions come fast. "How much space will the data need in five years?" "How much data must move each second?" "How many servers?" "What will it cost each month?"

A weak answer is "a lot of storage" or "add servers when needed". A strong answer multiplies a few numbers, writes the units, and finds the biggest item first. It also says what the bill is made of.

At SDE-1 level you should do each sum when the numbers are given. At SDE-2 level you should choose the numbers, and remember copies and spare room. At SDE-3 level you should also say which part of the bill is the largest, and what you would change to make it smaller.

Think about a cook who plans food for a wedding. The cook counts 500 guests and plans 400 g of food for each guest. That is 500 x 400 = 200,000 g, which is 200 kg. The cook adds some extra for guests who come late. Then the cook multiplies by the price per kilogram.

Capacity planning is the same. Capacity planning means working out ahead how much of each thing the system needs, with some room to spare. You do five sums. Use the busiest-time idea from lesson 14 for the sums where load matters.

Sum 1: storage

A row is one saved record, such as one post. Storage is:

• storage = rows per day x size of one row x days kept x copies

Retention is how long you keep the data. Copies are the extra versions of the data that are kept on different machines. If one disk breaks, another copy still has the data. A common choice is 3 copies. It is an assumption, so say it aloud.

Find the biggest item first. If one item is more than 99% of the total, you can ignore the small ones.

Units: 1 GB = 1,000 MB, 1 TB = 1,000 GB, and 1 petabyte (PB) = 1,000 TB.

Sum 2: bandwidth

Bandwidth is how much data moves through a link each second. Count the data coming in (uploads) and going out (downloads) separately.

• data per second = data per day / 86,400
• peak data per second = data per second x peak factor (lesson 14)

Watch the units. Storage is counted in bytes. Network speed is counted in bits per second: 1 byte = 8 bits. A speed of 1 Gbps (gigabit per second) is 1,000 Mbps (megabits per second). That is 1,000 / 8 = 125 MB per second.

Sum 3: how many servers

A CPU core is one worker inside the processor of a computer. CPU means central processing unit. One core can do 1,000 ms of work in 1 second. A request needs some ms of CPU time. This is the time the processor works, not the time spent waiting for the database. So:

• requests per second for one server = cores x 1,000 / CPU ms per request

A server with 4 cores and 8 ms of CPU per request does 4 x 1,000 / 8 = 500 requests per second. This is the same 500 requests per second that lesson 1 used.

Do not plan to use a server at 100%. Headroom is the room you leave unused, so a jump in traffic or a broken server does not stop the service. Choose a target such as 70% busy at peak. Then:

• servers = peak requests per second / (requests per second for one server x target busy share), rounded up, plus 1 spare server

Also check the network link of each server, which is its connection to the network. It must be large enough for the data the server sends out.

Sum 4: memory for popular data

Hot data is data that users read much more often than the rest, such as photos from the last two days. A read from memory takes about 100 ns. A read from an SSD takes about 150 µs (lesson 14). So if the hot data fits in memory, reads of it are much faster. To size it:

• memory needed = hot items x size of one item

Which data is hot is an assumption. State it. The number of machines is the memory needed divided by the memory of one machine, rounded up.

Sum 5: monthly cost

In lesson 13 you saw that you rent machines and pay for what you use. So the monthly bill has these parts:

• servers = number of servers x hours in the month x price per server-hour
• storage = GB stored x price per GB-month (one GB kept for one month)
• data out = GB sent out x price per GB

A 30-day month has 30 x 24 = 720 hours. Prices differ by provider and change often, so use an assumed price and say so.

We continue the photo app from lesson 14. It has 1 crore (10,000,000) daily users, 42 requests per user per day, and a peak of 14,583 requests per second.

Assumptions (made up to teach with, not real traffic or real prices):

• Of the 20,000,000 writes a day, 1 in 20 is a new photo post: 1,000,000 photos. The other 19,000,000 writes are likes and comments of 100 bytes each.
• A photo upload is 300 KB. A post record is 1 KB.
• 3 copies. Retention 5 years = 5 x 365 = 1,825 days (we ignore leap days). No growth in users.
• Of the 40 reads per user per day, 10 are photo views of 100 KB, and 30 are small answers of 5 KB.
• Peak factor 3 (from lesson 14).
• A server has 4 cores, uses 8 ms of CPU per request, and has a 1 Gbps network link. We plan for 70% busy.
• Price: Rs 10 per server-hour (as in lesson 13), Rs 2 per GB-month stored, Rs 8 per GB sent out. Data coming in is free. These are not real prices.

Sum 1: storage.

• Photos per day = 1,000,000 x 300 KB = 300,000,000 KB = 300 GB
• Post records = 1,000,000 x 1 KB = 1 GB
• Likes and comments = 19,000,000 x 100 bytes = 1,900,000,000 bytes = 1.9 GB
• Total per day = 300 + 1 + 1.9 = 302.9 GB. Photos are 300 / 302.9 = 99% of it.
• Five years: 302.9 GB x 1,825 = 552,792 GB, about 553 TB
• With 3 copies: 552,792 x 3 = 1,658,377 GB, about 1,658 TB, which is about 1.66 PB

For photos alone: 300 GB x 1,825 x 3 = 1,642,500 GB = 1,642.5 TB. This answers the predict box: about 1,000 TB.

Sum 2: bandwidth.

• In (photos only; they are 99% of the data): 300 GB per day = 300,000 MB / 86,400 = 3.47 MB per second = 27.8 megabits per second. At peak: 3.47 x 3 = 10.4 MB per second = 83 megabits per second.
• Out per user per day = 10 x 100 KB + 30 x 5 KB = 1,000 + 150 = 1,150 KB
• Out per day = 10,000,000 x 1,150 KB = 11,500,000,000 KB = 11,500 GB = 11.5 TB
• Out per second = 11,500,000 MB / 86,400 = 133 MB per second = 1,065 megabits per second, about 1.06 Gbps
• At peak: 1.06 x 3 = 3.19 Gbps

Data out is 11,500 / 300 = 38 times more than data in.

Sum 3: servers.

• One server: 4 x 1,000 / 8 = 500 requests per second. At 70%: 500 x 0.7 = 350.
• Peak 14,583 / 350 = 41.7, so 42 servers. Plus 1 spare = 43 servers.
• Network check: 43 servers x 1 Gbps x 0.7 = 30.1 Gbps. The peak is 3.19 Gbps, so the network is not the limit.

Sum 4: memory for hot data. Assumption: people mostly view photos from the last 2 days, and we keep the 100 KB version.

• Hot data = 2 days x 1,000,000 photos x 100 KB = 200,000,000 KB = 200 GB
• Assumption: one machine has 64 GB of memory. 200 / 64 = 3.1, so 4 machines.

Sum 5: monthly cost (at the end of year 5, when all the data is stored).

• App servers: 43 x 720 x Rs 10 = Rs 309,600
• Memory machines: 4 x 720 x Rs 10 = Rs 28,800
• Storage: 1,658,400 GB x Rs 2 = Rs 3,316,800 (rounded GB)
• Data out: 11.5 TB x 30 days = 345 TB = 345,000 GB. 345,000 x Rs 8 = Rs 2,760,000
• Total = 309,600 + 28,800 + 3,316,800 + 2,760,000 = Rs 6,415,200, which is about Rs 64 lakh (1 lakh = 100,000)

Servers and memory are only 338,400 / 6,415,200 = 5% of the bill. Storage is 52% and data out is 43%. Per daily user, the bill is 6,415,200 / 10,000,000 = Rs 0.64 a month.

• Planning solves a wrong-size system and a surprise bill. It costs accuracy and time, because every input is an assumption. I would redo it when the real numbers arrive.
• Headroom solves slow service when traffic is higher than planned or a server breaks. It costs more servers. At 100% busy we would need 14,583 / 500 = 29.2, so 30 servers plus 1 spare = 31. With headroom we need 43. That is 43 / 31 = 1.39 times more, which is 39% more. I would lower the headroom if traffic is steady and the bill matters.
• Three copies solve losing data when a disk breaks. They cost three times the storage. Two extra copies of the data cost 2 x 552,792 GB x Rs 2 = Rs 2,211,168 a month, about Rs 22 lakh. I would keep fewer copies of data that can be rebuilt, such as a smaller picture made from the original.

"Servers are the biggest cost." Here they are 5% of the bill. Storage and data sent out are 95%. Add up each part before you decide what to make smaller.

"Mbps and MB per second are the same." They differ by 8 times. Data in is 3.47 MB per second, which is 27.8 megabits per second. If you mix them, the network is wrong by 8 times.

"Storage is rows times size." That gives one day: 302.9 GB. You must also multiply by the days kept and by the copies: 1,825 x 3 = 5,475 times more, which is 1,658 TB.

1. Write the storage formula. How many ms of work can one CPU core do in one second?
2. A notes app stores 5 lakh (500,000) new notes a day, each 2 KB. It keeps 3 copies for 2 years (730 days). (a) How much storage does it need? (b) The peak is 4,167 requests per second. One server does 400 requests per second at 100% busy. We plan for 70% busy and one spare. How many servers?
3. In the photo app, your bill is Rs 3 lakh for servers and Rs 27.6 lakh for data sent out. A teammate suggests removing 10 of the 43 servers to save money. Each server costs Rs 7,200 a month. What do you say?