Token bucket vs sliding window rate limiting

The decision

Rate limiting sets the most requests a client may send in a period of time. The server refuses the extra requests with status 429 Too Many Requests. A token bucket holds up to B tokens. Tokens are added at a steady rate r, and each request takes one. A sliding window counts the requests in the last N seconds, measured back from now. You choose between allowing short bursts (bucket) and a strict count over any period (window).

Choose a token bucket when

• Clients send short bursts as normal use, such as an app screen that sends 8 requests at once.
• You want a simple rule: an average rate r and a burst size B.
• You want very little memory. The bucket keeps two numbers: the token count and the time of the last request.
• The service behind can take a short burst, but not a steady high rate.
• Public APIs use it: Stripe says it uses the token bucket algorithm and allows short bursts above the cap (see Sources).

Choose a sliding window when

• The limit must be strict over any period of time. For example, 120 per minute must mean 120 in any 60 seconds.
• A burst would hurt the service behind, or break a contract with a partner.
• Use the sliding window counter (two counters and a weight) when memory matters. Use the sliding window log (the time of every request) only when the limit is small and exactness matters.

The numbers that change the answer

Worked example at Indian scale. A shop app has an API. Assumptions: the rule is 120 requests per minute for each user, which is 2 per second. There are 1,000,000 users counted at once. Each stored number takes 8 bytes. The bucket has B = 20 and r = 2 tokens per second.

Fixed window problem. A user sends 120 requests at second 59 and 120 at second 61. Each window sees 120, so all 240 pass in 2 seconds. That is 2 times the rule.

Sliding window counter. Estimate = previous count x weight + current count. At second 15, the weight is (60 - 15) / 60 = 0.75. With 96 in the previous minute and 30 so far, this request is the 31st: 96 x 0.75 + 31 = 103, below 120, so it passes. With 48 so far, this request would be the 49th: 72 + 49 = 121, so it is refused. The estimate assumes the previous requests were evenly spread. Cloudflare tested this on 400 million requests and reported 0.003% wrongly allowed or limited (see Sources).

Token bucket. At most B + r x T requests pass in T seconds.

• 2 seconds: 20 + 2 x 2 = 24.
• 10 seconds: 20 + 2 x 10 = 40.
• 60 seconds: 20 + 2 x 60 = 140, which is 20 above the "120 per minute" rule.

An app screen that sends 8 requests passes, and 20 - 8 = 12 tokens are left. A script that sends 3 per second empties the bucket in 20 seconds. After that only 2 per second pass.

So the real choice is the burst: a bucket lets up to B requests pass at once, and a strict window does not.

What breaks with each choice

• Token bucket: the burst passes at full speed, and the limit over a minute is above the headline rate (140, not 120). If B is large, a burst can hurt the service. Tune B.
• Sliding window log: memory grows with the limit (960 MB here).
• Sliding window counter: the answer is an estimate.
• Both: the counters live in memory. With many servers, each server needs a shared store, or the limit is wrong by the number of servers. A limit by IP address punishes people who share one address. When refusing, return 429 with a Retry-After header, not 500.

What it costs

• The counter methods and the bucket use 16 MB for 1,000,000 users. The log uses 960 MB.
• A shared counter store adds a network call to each request. Check your latency budget.
• Changing method later is cheap, because clients see only 429. A stricter rule can break clients, so announce it. Plan for days (rough guess).

How each level answers

SDE-1: A token bucket allows bursts and refills at a steady rate. A sliding window counts requests in the last N seconds.

SDE-2: For a public API I use a token bucket, because real clients send bursts and it needs two numbers. For a strict contract, such as 120 per minute from a partner, I use a sliding window counter.

Senior: With B = 20 and r = 2, at most 140 requests pass in a minute. A strict window gives 120. The log needs 960 MB for 1,000,000 users, the counter 16 MB. The risk is many servers with local counters, so I use a shared store. The fallback is to limit per server when the store is down.

Staff: I ask what a burst costs the service behind us, and what a refused request costs the customer. I set one limit policy for the platform: limit by user ID, a burst size for each endpoint, and clear 429 messages. One team owns the limiter. I check the choice by the share of requests refused, the peak load on the service behind, and the number of customer complaints about limits.

Say it in the interview

I would use a token bucket keyed by user ID. I would set a burst of 20 and a refill of 2 per second, because apps send bursts. It allows at most 140 requests in a minute, against 120 for a strict window. For a partner who needs a strict limit, I would use a sliding window counter.

Common mistakes

• Using a fixed window and not seeing the edge problem. Twice the limit can pass in 2 seconds.
• Limiting by IP address only. Many people share one address.
• Keeping counters inside each server when there are many servers.
• Choosing B without thought. A big bucket means a big burst.

Sources

• Stripe, "Scaling your API with rate limiters": it uses the token bucket algorithm, and added a way to burst briefly above the cap for sudden spikes: https://stripe.com/blog/rate-limiters (opens in a new tab) (checked 26 Oct 2026).
• Sliding window counter: the formula and the 0.003% error on 400 million requests, reported by Cloudflare: https://blog.cloudflare.com/counting-things-a-lot-of-different-things/ (opens in a new tab) (checked 26 Oct 2026).
• User counts, limits and memory sizes are assumptions made for teaching.