Backpressure queue vs drop vs degrade

The decision

When work arrives faster than a service can finish it, something must give. Backpressure means the full part tells the part before it to slow down. You have three ways to handle the extra work. You can queue it, so it waits. You can drop it, so the request is refused or discarded. Or you can degrade, so the service gives a cheaper answer. Each choice is right for some kinds of work and wrong for others. Most real systems use all three, for different requests.

Choose a queue when

• The work is valuable and nobody is waiting on the screen, such as sending an email or building a report.
• The extra load is a short spike, and normal load leaves spare capacity to drain the backlog.
• The longest wait is shorter than the deadline of the work.
• The sender can wait or slow down, for example a nightly job or another internal service.

Choose to drop when

• The data is replaceable. The next reading, the next location ping or the next metric makes the lost one worthless.
• The request is old. Its caller has already given up, so finishing it is wasted work.
• The work is low priority, such as browsing during a sale, and the work that matters is checkout.
• A few losses are fine, and you can count them.

Choose to degrade when

• A person is waiting, and a cheaper answer is better than an error. Examples are a saved copy of a page, 10 results instead of 50, or no recommendations.
• A degraded path exists and costs much less than the full path.
• The service must stay up for everyone, and a lower quality for all is better than errors for some.

The numbers that change the answer

All numbers are assumptions. A service finishes 2,000 requests per second. Users give up after 5 seconds. A spike brings 3,000 per second for 60 seconds.

Queue with no limit. Assumption: the oldest request is served first. The backlog grows by 3,000 - 2,000 = 1,000 per second. A 5-second wait means a backlog of 2,000 x 5 = 10,000. That is reached after 10,000 / 1,000 = 10 seconds. After that, every request that finishes belongs to a user who has left. After 60 seconds the backlog is 60,000, and the last request waits 60,000 / 2,000 = 30 seconds. The requests that arrive in the first 10 seconds are the only ones served in time: 3,000 x 10 = 30,000 of 180,000.

Queue with a limit. Choose the size as capacity x the longest wait you accept (our own advice). With a 2-second wait: 2,000 x 2 = 4,000 requests. The queue is full after 4,000 / 1,000 = 4 seconds. Then 1,000 per second are refused: 56 x 1,000 = 56,000 of the 3,000 x 60 = 180,000 requests, which is 31 percent. Of the rest, 2,000 x 60 = 120,000 finish during the spike, each after a wait of at most 2 seconds. The last 4,000 finish just after. That is four times the 30,000 above (120,000 / 30,000).

When a queue is enough. Suppose normal load is 1,000 per second, so the spare capacity is 2,000 - 1,000 = 1,000 per second. A spike that leaves a backlog of 60,000 drains in 60,000 / 1,000 = 60 seconds, and the last request waits 30 seconds. If the deadline is 5 seconds, a queue is too slow. If the work is an email with a deadline of an hour, a queue is perfect.

Degrade. Assumption: the cheap path costs 0.25 of a full request. To serve 3,000 per second with capacity for 2,000 full requests, the average cost must be 2,000 / 3,000 = 0.667. If a share f of requests use the cheap path, then (1 - f) x 1 + f x 0.25 = 0.667. This gives 1 - 0.75 f = 0.667, so f = 0.333 / 0.75 = 0.444. About 44 percent of requests go to the cheap path, and nobody is refused. The cheap path alone can carry 2,000 / 0.25 = 8,000 per second. Above that, drop.

Drop by priority. If drops are random, each kind keeps 2,000 / 3,000 = 67 percent. If checkout is 10 percent (300 per second), priority keeps all 300. Browsing gets 2,000 - 300 = 1,700 of its 2,700, which is 63 percent.

What breaks with each choice

Queue.

• A long queue hides the problem. Wait grows, users leave, and the server does work for nobody. Bound the queue and drop the old items first.
• Memory. An unbounded queue can fill memory and crash the process. 60,000 requests of 10 KB (Assumption) is 600 MB.
• A restart loses the queue if it is only in memory. For work that matters, use a durable queue.

Drop.

• Refusals bring retries. A client that retries 3 times at once can turn 3,000 per second into up to 12,000. Send a Retry-After header, use random waiting and a retry budget.
• Silent loss. If nobody counts the drops, you do not know the damage. Count them and alarm.
• Wrong priority. If checkout is dropped with browsing, revenue falls. Decide priorities before the spike.

Degrade.

• The cheap path is untested. It fails on the day you need it. Test it every month.
• Stale or poor answers. A saved page may show a sold-out item. Put a limit on its age.
• It is not free. The degraded path still uses capacity (0.25 each), so there is a point where you must drop.

What it costs

All numbers here are assumptions.

• Queue: a durable queue service and a dead letter queue. A few days to add, plus alarms on queue age.
• Drop: a load shedding rule and a priority list. About a week, with product input on priorities.
• Degrade: a second code path for each feature, with a switch. One to three weeks for each feature, and a test each month.
• Changing your mind: these are rules, not data. Change them in a deploy. The slow part is agreeing on priorities.

How each level answers

The question is: "Traffic is 1.5 times capacity for one minute. What do you do?"

SDE-1: Put extra work in a queue, refuse some of it, or give a simpler answer. A queue with no limit only moves the delay.

SDE-2: I bound the queue and refuse the rest with 503 and Retry-After. Checkout has priority over browsing. For browsing I serve a saved page.

Senior: The backlog grows by 1,000 per second, so a queue of 4,000 fills in 4 seconds, and then 31 percent are refused. With a cheap path at 0.25 of the cost, only 44 percent degrade and nobody is refused. The failure is a retry storm, so I add a retry budget.

Staff: Product and engineering write one policy for each endpoint: what may wait, what may be lost, and what may be cheaper. The service teams own the policy, roll it out one endpoint at a time starting with checkout, and test it in a game day. We measure useful work (requests finished before the user left), drops for each priority, and the age of the oldest queued item.

Say it in the interview

For a short spike I would bound the queue at capacity times the longest wait, which is 4,000, and refuse the rest with 503. Checkout keeps priority, and browsing gets a cheaper saved page. The risk is a retry storm, so clients use random waiting and a retry budget.

Common mistakes

• "A bigger queue means no request is lost." The wait grows to 30 seconds while users give up after 5. The work is wasted.
• Dropping at random. Checkout loses 33 percent, the same as browsing. Drop by priority.
• Degrading without a limit on staleness. A saved page that is a day old can sell a product that is gone.
• Adding servers as the only plan. New servers take time to start, and the spike may be over by then.

Sources

• Google SRE book, "Handling Overload" (cheaper responses; shedding the less critical requests first; retry budgets): https://sre.google/sre-book/handling-overload/ (opens in a new tab) (checked 26 Oct 2026).
• Google SRE book, "Addressing Cascading Failures" (bounded queues, load shedding, graceful degradation, retry budgets): https://sre.google/sre-book/addressing-cascading-failures/ (opens in a new tab) (checked 26 Oct 2026).
• The rates, times, costs and message sizes are assumptions made for teaching. The queue size rule (capacity x longest wait) is our own advice. The other numbers are calculations shown above.