Reactive vs predictive autoscaling vs over-provisioning

The decision

Autoscaling changes the number of servers to fit the load. Reactive autoscaling watches a measure such as CPU use and adds servers after the load has risen. Predictive autoscaling uses past traffic to forecast the load and adds servers before it comes. Over-provisioning means you keep enough servers for the peak all the time and do not scale. The three differ in cost when load is low, and in failed requests when load jumps. Reactive is cheap but slow. Over-provisioning is safe but wasteful. Predictive sits between them and works only when the pattern repeats.

Choose reactive autoscaling when

• Load changes slowly, or in small steps, so your spare room covers the lag of a few minutes.
• Traffic has no pattern you can forecast, such as news or a viral post.
• Servers start fast: seconds, not minutes.
• A few minutes of slow responses cost little, for example an internal tool.

Choose predictive autoscaling when

• The same shape repeats every day or week: office hours, lunch, 8 pm food orders, a match at a fixed time.
• Servers take minutes to start, or an application takes minutes to warm up, so a reactive scale-up arrives late.
• You have at least two weeks of clean history for that service.
• A forecast that is too high costs a little, and too low costs more.

Choose over-provisioning when

• Failing for even two or three minutes costs more than the idle servers. Payments and ticket sales on sale day are examples.
• The spike is sudden and unpredictable, and a forecast cannot help.
• The fleet is small, so the spare capacity is a small bill.
• Scaling needs something that is slow or can run out, such as database connections or a quota.

The numbers that change the answer

All of these are Assumptions unless marked as cited.

The lag of a reactive scale-up (calculation). Metrics arrive after 30 s. The autoscaler checks every 15 s (the Kubernetes default, cited). A new node takes 90 s. A new pod starts in 45 s. Total: 30 + 15 + 90 + 45 = 180 s, or 3 minutes. This is a best case. A large jump may need several scaling steps, so the real gap can be longer.

A spike (assumptions and calculation). 100 pods, each handling 200 requests a second. Normal load is 14,000 requests a second, so the pods run at 70 percent. At 8:00 pm load rises to 42,000 requests a second in 60 s, which needs 42,000 / (200 x 0.7) = 300 pods. Existing capacity is 100 x 200 = 20,000.

• Load passes 20,000 after 60 x (20,000 - 14,000) / (42,000 - 14,000) = 12.9 s.
• The excess grows to 22,000 by second 60 and stays there until second 180.
• Requests that cannot be served: the ramp is 22,000 x 47.1 / 2 = 0.52 million. The plateau is 22,000 x 120 = 2.64 million. The total is about 3.16 million.
• At Rs 0.5 of value per request, that is about Rs 15.8 lakh (one lakh is 100,000) per event.

Cost of each option (assumptions). One pod costs Rs 3 an hour.

• Over-provision 300 pods all month: extra 200 pods x 24 x 30 = 144,000 pod-hours, which is Rs 4.32 lakh a month.
• Predictive: extra 200 pods for 15 minutes a day: 200 x 0.25 x 30 = 1,500 pod-hours, which is Rs 4,500 a month.
• Reactive: no extra cost, but Rs 15.8 lakh per event.

Break-even for events (calculation). If one such spike comes every month, reactive loses Rs 15.8 lakh against Rs 4.32 lakh for over-provisioning, so the spare servers win. If one comes each quarter, the loss is Rs 5.3 lakh a month (15.8 / 3), which is still more. If one comes twice a year, the loss is Rs 2.6 lakh a month (15.8 x 2 / 12), and reactive wins. The break-even is 4.32 / 15.8 = 0.27 spikes a month, about one every 3.7 months. Predictive wins over both when the event repeats at a fixed time. All of this depends on the Rs 0.5 value per request, which is an Assumption, so test your own.

What the tools do (cited). The Kubernetes autoscaler for pods computes desired replicas as the current replicas times the current metric divided by the target, rounded up. It checks every 15 s by default and ignores changes within a 10 percent tolerance. It waits a 300 s stabilisation window before it scales down. AWS predictive scaling needs 24 hours of data and works best with two weeks. It forecasts hourly for the next 48 hours and refreshes every 6 hours. It starts in a mode that only forecasts. It does not scale in, so you still need a reactive policy for that. By default it scales at the start of each hour, and a pre-launch setting starts earlier (see Sources).

What breaks with each choice

Reactive.

• The lag leaves a gap in which users see errors. Fallback: keep spare room (target 60 to 70 percent busy), shed load, and use queues.
• Scaling flaps up and down. Fallback: stabilisation windows and a tolerance.

Predictive.

• A pattern breaks: a holiday, a sale, a new feature. Fallback: run a reactive policy together, because the larger of the two wins in AWS.
• A wrong forecast wastes money or misses the peak. Fallback: use forecast-only mode first and compare for two weeks.

Over-provisioning.

• The bill is high all month for a few busy minutes. Fallback: scale down outside known windows.
• Other work quietly uses the spare capacity. Fallback: alarms for headroom.
• A larger peak next year breaks the number. Fallback: load tests each quarter.

What it costs

• Money: Rs 4.32 lakh a month, Rs 4,500 a month and Rs 0 extra, from the example above.
• People (Assumption): reactive takes 1 to 2 engineer-days with load tests. Predictive adds about a week to check the forecast. Over-provisioning takes hours, then a quarterly load test.
• Changing your mind: all three are settings, so you can move in days. The hard part is to know which one you need, so log every spike and its lost requests.

How each level answers

SDE-1: Autoscaling adds servers when load rises. Predictive adds them before. Over-provisioning keeps extra servers running.

SDE-2: I start with reactive scaling on CPU at about 60 to 70 percent. For a known evening peak I add a schedule, so servers are ready before it.

Senior: A reactive scale-up lags 180 s. In that time a 3x spike leaves about 3.2 million requests unserved, worth about Rs 15.8 lakh. A daily event is cheap to pre-scale, at Rs 4,500 a month. A rare event is cheaper to absorb than to hold 200 spare pods at Rs 4.32 lakh a month. The break-even is about one spike every 3.7 months. I run reactive and predictive together, and I add load shedding for what is left.

Staff: I ask product how often the spike comes and what a failed minute costs. Then I pick the cheapest plan that meets that. I keep a spare floor only for the revenue path. Each service owner reviews forecast accuracy. I measure unserved requests, cost per request and time to reach the target. I start with reactive. I add predictive in forecast-only mode when a pattern appears. I hold extra capacity only when events cost more than the spare servers. I revisit after each big event.

Say it in the interview

I would start with reactive scaling at 65 percent CPU, because it is simple and follows load down as well as up. For the 8 pm peak, which repeats daily, I would add a forecast or schedule so servers are ready 15 minutes early. In my numbers that costs about Rs 4,500 a month, not Rs 4.32 lakh for always-on spare. The risk is a spike nobody forecast, so I keep the reactive policy and add load shedding for the 3-minute gap.

Common mistakes

• Trusting CPU alone. If the limit is a database connection or a queue, CPU stays low while users fail.
• Ignoring start-up time. A node and pod that take 135 s mean the scale-up is late by design.
• Forecasting without a reactive backup. The first break in the pattern becomes an outage.
• Holding spare capacity for every service. Only the revenue path needs it.

Sources

• Kubernetes documentation, Horizontal Pod Autoscaling (formula, 15 second default sync period, 0.1 tolerance, 300 second scale-down stabilisation window): https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/ (opens in a new tab) (checked 26 Oct 2026).
• AWS, "How predictive scaling works" (24 hours minimum, up to 14 days analysed, hourly forecast for 48 hours, refreshed every 6 hours, forecast-only mode, no scale-in, scaling at the start of each hour unless pre-launch is set, the larger desired capacity wins when policies are combined): https://docs.aws.amazon.com/autoscaling/ec2/userguide/predictive-scaling-policy-overview.html (opens in a new tab) (checked 26 Oct 2026).
• Pod size, load, start-up times, value per request, pod price and event frequency are assumptions made for teaching. The lag, the lost requests and the break-even figures are calculations.