Batch vs real-time processing

The decision

A batch job runs now and then. It reads a stored pile of data, such as yesterday's events, and writes results. A stream job runs all the time. It reads each new event as it arrives and updates the results a little at a time. Batch gives exact results, late. Stream gives quick results that may be less exact. Many systems use both.

Choose batch when

• A result that is hours old is fine, such as a daily report, a bill or a salary run.
• You need exact numbers, including events that arrive late.
• You want to run the job again after a bug fix. Batch reads the stored events again.
• Cost matters. A batch job uses servers only while it runs.
• The logic is complex, such as joining large tables.

Choose real-time when

• A person or a program must act within seconds, such as fraud checks, live dashboards or alerts.
• The value of the answer falls fast with time. A late warning is useless.
• The data never stops, and one big daily run would be too large or too slow.
• A small error is acceptable, because you fix it later with a batch.

The numbers that change the answer

Worked example at Indian scale. An ad network counts clicks. Assumptions: 2,000,000,000 clicks a day, each event 200 bytes. One stream job runs on one server and handles 10,000 events per second, planned at 70% load. The peak is 3 times the average. A click is worth Rs 5. Of all clicks, 0.1% arrive after the stream's wait time of 2 minutes.

The batch read takes 200 seconds. Assumption: the whole nightly job, with counting and writing, takes 30 minutes.

Cost of always on. Assume the stream is sized for the peak and runs 24 hours: 10 jobs x 24 hours = 240 server hours a day. The batch with 20 workers runs 0.5 hour: 20 x 0.5 = 10 server hours a day. The stream uses 240 / 10 = 24 times more server time, in return for results within seconds.

Cost of being late. Late clicks: 0.1% of 2,000,000,000 = 2,000,000 clicks. At Rs 5 each that is Rs 10,000,000 (1 crore) a day. The stream alone would miss it, because it closes each minute after 2 minutes. The nightly batch counts them, so the bill is exact.

Freshness. A stream answer is ready in seconds. A nightly batch answer is up to 24 hours old. A batch every hour is up to 1 hour old, but runs 24 times a day.

A common design uses both: the stream for the live chart, the batch for the bill.

What breaks with each choice

• Batch: results are old. A job that fails at 3 am means no report at 9 am. Data grows, so a job that took 30 minutes can take 3 hours. Fix: alerts on job time, and smaller, more frequent runs.
• Real-time: the job is always on, so it needs always-on care. If it falls behind, the lag grows. A crash can count events twice, so the job must save its position and its counts together. Events can arrive late or out of order, so you need a rule such as "wait 2 minutes, then close the minute".
• Both: two programs must give the same answer. They need the same counting rules, or the live chart and the bill will disagree.

What it costs

• Batch is the cheapest to build and to run: 10 server hours a day in our example.
• Real-time costs 24 times the server time here, plus more engineering: window rules, late events, restarts (weeks, rough guess).
• Moving later: you can add a stream to a batch system by reading the same stored events. Replacing a stream with batch is simple, but users lose fresh numbers.

How each level answers

SDE-1: Batch runs on a schedule over stored data. Stream processes each event as it arrives.

SDE-2: For a monthly bill I use batch, because exactness matters and a day's delay is fine. For a fraud alert I use a stream, because the answer is useless after a minute.

Senior: At 69,444 events per second I need 10 stream jobs. The stream misses 2,000,000 late clicks a day, Rs 10,000,000, so I run a nightly batch from the stored events for the bill. The risk is that the two disagree, so they share the counting rules. The fallback is to run the batch again.

Staff: I ask what each hour of delay costs, and what a wrong number costs. I start with batch, since it is cheap and correct, and add a stream only for screens or alerts that need seconds. One team owns the counting rules, and a check compares stream and batch totals each night. I measure freshness, the gap between the two totals, and the cost per million events.

Say it in the interview

I would use a stream for the live chart and a nightly batch for the bill. The stream needs about 10 jobs at the 69,444-per-second peak. The batch recounts from stored events and catches the 0.1% of clicks that arrive late, which is Rs 1 crore a day.

Common mistakes

• Choosing real-time because it sounds modern. Always-on costs 24 times the server time here.
• Using the time the job read the event, not the time the event happened. A lagging job then puts clicks in the wrong minute.
• Letting stream and batch use different rules, so totals never match.
• Making a bill from the stream alone. Late events are missed.

Sources

• All traffic numbers, sizes, job speeds, the 30-minute batch time and the Rs 5 per click are assumptions made for teaching. Server hours are calculated from them.