Cache stampede lock vs early refresh vs serve stale

The decision

A cache stampede happens when a popular entry expires and many requests miss it at the same moment. Each request runs the slow load, so the database gets many copies of the same query. Three fixes are common. A lock lets one request load the value while the others wait. Early refresh reloads the value shortly before it expires, so it never goes missing. Serve stale returns the old value at once while one request loads a new one in the background.

Choose a lock when

• A user can wait about one load time, and the value must never be older than its time to live (TTL).
• The entry is missing for real, for example after a restart. There is no old value to serve.
• Many different keys miss, each with few readers. A lock for each key costs little.
• The load is so costly that even a few duplicate loads hurt.

Choose early refresh when

• A few keys are very hot, and no reader should ever wait.
• Data must stay fresh, and a small extra load shortly before expiry is acceptable.
• The reads themselves can trigger the refresh, so keys that nobody reads are not refreshed.
• You know the load time, so you can set the early window from it.

Choose serve stale when

• A slightly old answer is better than a wait or an error, as with a product page, a score or a list of deals.
• The loader may be slow or may fail, and old data keeps the page alive.
• You can state the age that users accept.
• The read path must stay fast even while the database struggles.

You can combine them. A common pattern is serve stale to readers, plus one lock for the single background refresh.

The numbers that change the answer

All numbers are assumptions. One hot key gets 200 reads per second. A load takes 1.5 seconds when the database is healthy. The TTL is 300 seconds.

No protection. At expiry, the reads that arrive during the load also miss: 200 x 1.5 = 300. That is 1 + 300 = 301 loads of the same value. It is a lower bound, because 301 loads slow the database, and slower loads bring more misses.

Lock. One request loads, and the other 300 wait at most 1.5 seconds (0.75 seconds on average). In one TTL the key gets 200 x 300 = 60,000 reads, so 300 / 60,000 = 0.5 percent of reads wait. Give the lock a time limit of 3 seconds, so a crashed holder frees it. Then up to 200 x 3 = 600 requests may be held at once.

Early refresh. The XFetch rule from a research paper (Vattani and others, 2015) is simple. On each read, reload the key if the current time minus delta x beta x ln(random number) is at least the expiry time. Here delta is the load time (1.5 seconds) and beta is a tuning number that the paper sets to 1. The ln of a number between 0 and 1 is negative, so each read looks ahead by a random time, 1.5 seconds on average.

With 200 reads per second, 300 reads arrive during one load. The first reload is expected about 1.5 x ln(300) = 1.5 x 5.7 = 8.6 seconds before expiry. Readers keep getting the old value during the reload, so nobody waits.

How many loads? This is our own calculation, not the paper's. Assume no marker for "a reload is running", so reads during the 1.5-second reload can also pass the test. Then the expected number of loads is 1 + (e - 1) = e, about 2.7, against 301. A Python simulation of 2,000 expiry cycles with random arrivals gave 2.65 to 2.76 in three runs. With a marker that lets only one reload run, it gave 1.0. The paper's own model reports a similar small number, about 2.4 processes.

Serve stale. Assumption: a stale window of 60 seconds after the 300-second TTL. All 200 reads per second get an answer at once, and one request loads in the background. The oldest data a user sees is 300 + 60 = 360 seconds old.

Cold start. Assumption: 4,000 reads per second with a 95 percent hit rate, so 4,000 x 0.05 = 200 reads per second reach the database. An empty cache sends all 4,000, which is 4,000 / 200 = 20 times more. There is no old value, so early refresh and stale help nothing. Lock each key, and warm the popular keys before users return.

Entries that expire together. Add random jitter of plus or minus 25 percent to the TTL. Ten hot keys that were filled together then expire over 300 x 0.5 = 150 seconds, from 225 to 375 seconds.

What breaks with each choice

Lock.

• Waiting requests hold connections. If the load fails, they all fail together.
• A per-server lock stops duplicates only inside one server. With 20 servers you can still get 20 loads. A shared lock needs one atomic "set if absent" step in the cache.

Early refresh.

• Reads trigger it, so the first reads after an idle period still miss. It does not help a cold key.
• A refresh that fails leaves the old value until expiry. Then the miss and the stampede return.

Serve stale.

• Users see old data. For a price or stock count this may be wrong, so choose the window for each kind of data.
• A refresh that keeps failing serves old data forever unless you bound the age.
• Nothing is stale on a cold start. Pair it with a lock.

What it costs

All numbers here are assumptions.

• Lock: a few hours of work. Users pay with waiting time.
• Early refresh: about a day. The cost is a few extra loads, 2.7 per expiry instead of 1 (or none extra with a marker).
• Serve stale: about a day, plus a decision on the stale window for each kind of data. The cost is data age.
• Changing your mind: all three are code in the read path, with no data to move. A change is a deploy, and a flag can switch it on for one key family.

How each level answers

The question is: "A popular product page expires and the database spikes. Fix it."

SDE-1: A stampede is many requests loading the same missing entry at once. One fix is to let only one request load it.

SDE-2: I add a lock for each key, so one request loads and the others wait, and I add jitter to the TTL. For a page that can be a little old, I serve the stale value while one request reloads.

Senior: An unprotected expiry makes 301 loads. A lock makes 1 and delays 0.5 percent of reads by up to 1.5 seconds, and early refresh makes about 3 with no waiting. I serve stale with a 60-second window plus one lock for the refresh, and I warm the cache after a restart.

Staff: I agree a freshness rule with product for each data type, such as how old a price may be. The platform team ships one shared cache library with these options. We measure load at expiry, waiting reads and data age, and we test a full cache flush before a big sale.

Say it in the interview

For a hot page I would serve the stale value for up to 60 seconds while one request reloads it under a lock. The database then gets 1 load instead of 301 and nobody waits. Stale data does not help a cold cache, so I add jitter and warm the popular keys, and I bound the age.

Common mistakes

• Using only a longer TTL. A longer TTL makes stampedes rarer, not smaller. There are still 301 loads at expiry.
• Using a lock for a cold cache of many keys. A lock removes duplicates for one key. It does not reduce the load of many different keys.
• Serving stale with no age limit. A failing refresh then serves old data forever.
• Refreshing early with a timer for every key. You reload keys that nobody reads. Let reads trigger the refresh.

Sources

• Vattani, Chierichetti and Lowenstein, "Optimal Probabilistic Cache Stampede Prevention", VLDB 2015 (the XFetch rule; delta is the recompute time; beta is 1 by default; a stampede of about 2.4 processes in its model): https://www.vldb.org/pvldb/vol8/p886-vattani.pdf (opens in a new tab) (checked 26 Oct 2026).
• RFC 5861, "HTTP Cache-Control Extensions for Stale Content" (stale-while-revalidate and stale-if-error; these are HTTP headers, and the idea is the same in an application cache): https://www.rfc-editor.org/rfc/rfc5861 (opens in a new tab) (checked 26 Oct 2026).
• The read rates, load time, TTL, windows and costs are assumptions made for teaching. The 2.7 loads is our own calculation, checked by our own simulation. It differs from the paper's 2.4 because the paper's model uses different read arrival and reload-time assumptions than our exercise. The other numbers are calculations shown above.