Skip to content

Trade-offs · Senior and Staff

Cell-based architecture vs one shared system

When to split a system into independent cells that each serve a slice of customers, and when one shared system is the better buy, with the numbers.

Asked at: SeniorStaff

The decision

In a shared system, all customers use one set of servers and one database. A bad change or a bad request can hurt everyone. In a cell-based architecture, you run many complete copies of the system. Each copy is a cell. A cell serves a fixed group of customers and shares no state with other cells. A thin cell router sends each request to the right cell using a key such as the customer ID. A failure then hurts one cell, not all. You pay with more things to run, and with a router that every request must pass.

Choose one shared system when

  • Revenue per hour is low, so one bad hour costs less than the extra bill and the extra work.
  • The team is small, and every cell needs deploys, alarms and data checks.
  • Most requests touch many customers, such as a global feed or a marketplace search.
  • You have no safe key for splitting. A wrong key forces cross-cell calls.
  • A canary deploy on the shared system already catches most of your incidents.

Choose cells when

  • One failure has hit all customers more than once, at a large cost in money, trust or contract penalties.
  • The causes are things a canary does not catch. Examples are a poison request that crashes servers, one customer exhausting a limit, bad data, or a bad configuration that spreads at once.
  • A natural key exists (customer, tenant, region) and few requests cross it.
  • Customers or contracts ask for isolation, or you must split anyway because of a limit such as database size.

The numbers that change the answer

Blast radius (calculation). If N cells each serve an equal share, one failed cell affects 1/N of customers. That is 50 percent for N = 2, 25 percent for 4, and 5 percent for 20. AWS gives an example: with 10 cells serving 100 requests, a failure in one cell leaves 90 percent unaffected.

Does it pay? (assumptions). Revenue is Rs 5 lakh an hour (one lakh is 100,000). Assumption: 4 incidents a year take the shared system down for 1 hour each, and a canary does not catch them. With cells, an incident stays in one cell, and a person drains that cell in 20 minutes (one third of an hour). Each cell adds Rs 40,000 a month of duplicate fixed cost. We count this cost for every cell, which is a simple and slightly high choice.

  • Shared: 4 x 1 x 5 = Rs 20 lakh a year lost.
  • 4 cells: 4 x (1/3) x 25 percent x 5 = Rs 1.67 lakh lost. Saving: 20 - 1.67 = Rs 18.3 lakh. Extra bill: 4 x 0.4 x 12 = Rs 19.2 lakh a year.
  • 20 cells: 4 x (1/3) x 5 percent x 5 = Rs 0.33 lakh lost. Saving: Rs 19.7 lakh. Extra bill: 20 x 0.4 x 12 = Rs 96 lakh a year.

Break-even revenue is the extra bill divided by the saving per lakh of hourly revenue. For 4 cells: 19.2 / (18.3 / 5) = Rs 5.2 lakh an hour, close to today. For 20 cells: 96 / (19.7 / 5) = Rs 24.4 lakh an hour. So at this revenue, 4 cells are near break-even and 20 cells lose money.

Read this model with care. The result for 4 cells is a tie, and small changes flip it. Count only the 3 extra cells. The bill is 3 x 0.4 x 12 = Rs 14.4 lakh, so 4 cells save 18.3 - 14.4 = Rs 3.9 lakh a year. With 2 incidents a year, not 4, 4 cells lose again. So money does not decide this; risk does. The questions are how often a canary misses a failure, how bad a total outage is for trust, and whether a customer demands isolation.

The router limits you (calculation). Every request passes the router. A router at 99.99 percent allows 43,200 x 0.0001 = 4.3 minutes of downtime a month, whatever the cells do. AWS calls it the thinnest possible layer; keep it that way.

Cell size. AWS names three forces. A cell must be big enough for your largest customer, small enough to test at full size, and big enough for savings from scale. Assumption: your largest customer sends 8 percent of traffic. A cell of 5 percent cannot hold that customer, so each cell must hold at least 8 percent. That allows at most 12 cells (100 / 8 = 12.5), so 20 cells would not work.

Cost is not always a multiple. AWS notes that 30 hosts can stay 30 hosts, grouped into cells. The extra cost is the fixed part of each cell.

What breaks with each choice

Shared system.

  • One bad deploy, one poison request or one heavy customer hurts everyone. Fallback: canary deploys, per-customer limits, and a bulkhead around each dependency.
  • You hit a scaling limit all at once. Fallback: load tests, and split before the limit.

Cells.

  • A request needs two cells, such as a transfer between two customers. Fallback: choose a key that makes this rare.
  • The router fails, or its mapping is wrong, and every customer is down. Fallback: keep the mapping small and cached, and run the router in many places.
  • The cells drift into different versions and settings. Fallback: one pipeline that deploys cell by cell.
  • One customer grows and the cell sizes become uneven. Fallback: move customers between cells with a rehearsed control plane.

What it costs

  • Money: Rs 40,000 a month for each cell (Assumption), so 4 cells add Rs 1.6 lakh a month and 20 cells add Rs 8 lakh.
  • People (Assumption): 2 engineers for 3 to 4 months build the router, control plane and pipeline. Then 1 engineer runs them.
  • Changing your mind: shared to cells takes months, because data must be split by the key. Going back is easier for the code, but moving the data is also slow. Add the key to every request and table first, so you can go either way.

How each level answers

SDE-1: Cells are copies of the system. If one fails, the others work.

SDE-2: I split customers into cells behind a router, so a bad deploy hurts few customers. The cost is more servers and more operations work.

Senior: With 20 cells, a failure hits 5 percent, but the extra Rs 96 lakh a year does not pay. Four cells are a tie in money, so I decide on risk: how often a canary misses a failure. I start with 2 to 4 cells and deploy one cell at a time. I check that the largest customer, at 8 percent, fits in a cell. The router stays thin, because it caps availability at about 4.3 minutes a month.

Staff: I ask what a total outage costs and how often a canary cannot catch the cause. Product and sales tell me which customers need isolation, and one platform team owns the router. I plan the path. First put the partition key in every request and table, and deploy in waves on the shared system. Then split into 2 cells, then 4. I move on when the yearly loss that cells would remove is larger than the yearly extra bill, or when a contract asks. I know it was right when incidents stay inside one cell. I revisit it when cell sizes become uneven.

Say it in the interview

I would keep one shared system now and add the customer ID to every request and table, so we can split later. We deploy in small waves so that a bad change reaches 5 percent first. I would move to 4 cells behind a thin router if cells would remove over Rs 19 lakh a year of loss. A big customer who needs isolation is the other reason. The risk is a request that needs two cells, so I would pick the key to make that rare.

Common mistakes

  • Treating cells as a way to scale only. The main gain is a small blast radius. Test that a failed cell stays alone.
  • Putting business logic in the router. All cells share it, so a bug there takes down every cell.
  • Choosing a key that cuts across requests. If most requests touch two cells, you rebuilt a shared system with extra steps.
  • Skipping the canary. A canary on a shared system gives much of the deploy protection for far less money.

Sources

Understand the trade-off. Practise explaining your choice.

In an AI mock interview, the interviewer can push on a choice like this one: it changes a number and asks if your answer still holds.