Hash vs range vs directory sharding

The decision

Sharding splits the data of one database over several servers, so that each server (a shard) holds a part. The shard key is the column that decides which shard owns a row. There are three ways to turn a key into a shard. Hash sharding computes a hash of the key and uses it to pick the shard. Range sharding gives each shard a continuous range of key values. Directory sharding keeps a lookup table that lists the shard of each key (or each group of keys). The choice decides how even the load is, which queries stay on one shard, and how hard it is to add servers.

Choose hash sharding when

• Most reads and writes are by one key: get a user, get an order, get a cart.
• You need an even spread, and the key is an ID with no useful order.
• Writes arrive in time order. A range on time would send every write to one shard.
• You can live with scatter-gather for the rare range query. A scatter-gather query asks every shard and merges the answers.

Choose range sharding when

• The main query reads a continuous range, such as one device over a day, or orders from 1 to 7 October.
• You can build the key so that writes spread. For example, use device ID first and time second.
• You want to split one busy range in two without touching the others.

Choose directory sharding when

• Some customers are very large. One tenant may need its own shard.
• You must place data by rule, such as "customers in the EU stay on EU servers".
• You can run a small, always-available lookup service and cache it.

The numbers that change the answer

Assumptions:

• Total data is 2 TB (2,000 GB), in 4 shards of 500 GB each.
• Copy speed is 100 MB per second.
• One shard takes 700 writes per second. The app sees 2,000 writes per second.

Even spread (calculation).

• Range on date: all new rows go to the newest shard. It must take 2,000 writes per second but can take 700. That is 2,000 / 700 = 2.9 times too busy, while the other three sit idle.
• Hash on order ID: 2,000 / 4 = 500 writes per shard. That is 500 / 700 = 71% busy.

Adding a shard, 4 to 5 (calculation).

• Hash mod number of shards: N / (N + 1) = 4 / 5 = 80% of rows move. That is 0.8 x 2,000 GB = 1,600 GB. Time = 1,600,000 MB / 100 MB per second = 16,000 seconds = 4.4 hours.
• Hash with 1,024 fixed buckets mapped to shards by a small table: move only the buckets for the new shard. That is 1/5 of the data = 400 GB = 4,000 seconds = 1.1 hours.
• Range: split the largest range and move half of one shard = 250 GB. Time = 2,500 seconds = 42 minutes.
• Directory: move the tenants you pick. Moving 400 GB takes the same 1.1 hours.

Scatter-gather cost (calculation). Assumption: each shard answers slowly (over 100 ms) for 1% of requests. A query on all shards is slow if any shard is slow. With 4 shards: 1 - 0.99 to the power of 4 = 3.9%. With 16 shards: 1 - 0.99 to the power of 16 = 14.9%. More shards make range queries on a hash key slower on average.

Directory size (calculation). Assumption: 10,000,000 (1 crore) users, 16 bytes per entry (an 8-byte user ID and a shard number). Size = 10,000,000 x 16 = 160,000,000 bytes = 160 MB. It fits in memory on every app server. If every request needs it and it is up 99.9% of the time, it adds 0.001 x 43,200 = 43.2 minutes of downtime a month. So cache it.

Cited facts. MongoDB says hashed sharding spreads data more evenly, and range queries on a hashed key likely go to many shards. Redis Cluster uses 16,384 hash slots, which is the fixed-bucket idea above (see Sources).

What breaks with each choice

Hash.

• With a plain "mod N", adding a shard moves 80% of the rows. Fallback: use fixed buckets or consistent hashing.
• One very busy key still lands on one shard. Fallback: add a small suffix to the key and spread it, or give it its own shard.
• Range queries become scatter-gather, and get worse as shards grow.

Range.

• A key that only grows sends all new writes to one shard. Fallback: put a spread value first in the key, or use hash for the write path.
• Ranges become uneven as data grows. Fallback: split a range automatically when it passes a size. Watch for many splits at once, because each one moves data.

Directory.

• The directory is one more part that must be fast and up. Fallback: cache it in each app server, and keep a copy with followers.
• A stale cache sends a request to the old shard during a move. Fallback: the old shard answers "moved" and the app refreshes the entry.
• A move must not lose writes. Fallback: copy, write to both for a short time, switch, then delete the old rows.

What it costs

• Servers. Each shard needs followers. Assumption: 4 shards with 2 followers each is 4 x 3 = 12 servers, against 3 for one unsharded database.
• People. A directory is a new service. Assumption: 3 to 6 engineer-weeks. Hash and range come with the database.
• Changing your mind. Moving from one scheme to another means re-keying all the data. The copy takes 2,000,000 / 100 = 20,000 seconds = 5.6 hours. The project takes weeks: dual writes, a check that both sides match, and a switch with a way back. The key is the hardest thing to change.

How each level answers

SDE-1: Hash spreads rows evenly, range keeps nearby keys together, and a directory is a table that says where each key lives.

SDE-2: I shard orders by hash of the order ID, because a date range would put all writes on one shard. For date-range reports I keep a separate store.

Senior: At 2,000 writes per second, a date range is 2.9 times too busy on one shard, and hash gives 71% on each of 4. A fifth shard with plain mod moves 80% of 2 TB, about 4.4 hours, so I use 1,024 fixed buckets and move 1/5. Range queries on a hash key touch all shards, so I keep them rare.

Staff: I ask which customers are large and which rules apply by law, because that decides if a directory is worth a new service. The first version uses hash with fixed buckets, which is easy to run. I add a directory only when one tenant passes a set share of traffic (Assumption: 20 percent) or when data residency forces it. I track the hottest shard against the average, the share of scatter-gather queries and the time to add a shard. The data platform team owns the directory and the move tool.

Say it in the interview

I would shard orders by a hash of the order ID, split into 1,024 fixed buckets. At 2,000 writes per second that gives about 500 per shard. A date range would put all of it on one shard. Adding a shard moves a fifth of the data, not four fifths. If one big customer dominates, I would add a small directory for it.

Common mistakes

• Sharding by date because "reports use dates". All writes hit the newest shard. Spread the writes, and serve reports from a separate copy.
• Using "hash mod N" and not counting the move. Going from 4 to 5 shards moves 80% of the data.
• Picking a key with few values, such as city. One large city fills a shard, and the key cannot spread it.

Sources

• MongoDB manual, hashed sharding: more even data distribution, but range queries on a hashed key likely need broadcast operations to many shards. https://www.mongodb.com/docs/manual/core/hashed-sharding/ (opens in a new tab) (checked 26 Oct 2026)
• Redis Cluster specification: the key space is split into 16,384 hash slots. https://redis.io/docs/latest/operate/oss_and_stack/reference/cluster-spec/ (opens in a new tab) (checked 26 Oct 2026)
• Sizes, speeds and rates above are assumptions.