Tracks · SDE-2, Senior
AI engineering system design
An AI feature is a normal system with one strange part: a model that is slow, costs money for every word, and can be wrong. This track teaches that part in order. You learn tokens, speed and cost, how to serve an answer as a stream with a fallback, how to let the model read your documents, how to check that answers are good and safe, and how to keep the bill under control. Then you practise on AI design problems.
- 23steps in 5 stages
- 7 h 20 minin total
- 2 weeksat 4 hours a week
Start here if you know normal backend design, and you want to add the AI part.
Do the SDE-1 or SDE-2 track first if caches, queues and retries are still new to you.
The track has 11 lessons, 4 trade-offs, 8 practice steps. Steps marked “Free account” are inside the app.
Stage 1 of 5 · 1 h 12 min
Three basics to know first
You know caching, retries and rate limits, because every AI design uses them.
- 24 min
Lesson
Caching basics: hit, miss, TTL, hit rate
Why a cache protects the database, and how to estimate how much it helps with simple arithmetic.
- 24 min
Lesson
Timeouts, retries, backoff and jitter
How to set a time limit on every call, when to try again, how long to wait between tries, and how retries can turn a small problem into a storm.
- 24 min
Lesson
Rate limiting: token bucket, leaky bucket, sliding window
How to limit the requests one user can send, with five counting methods (fixed window, sliding window log, sliding window counter, token bucket, leaky bucket) and what each one costs.
Stage 2 of 5 · 1 h 41 min
The model as a system part
You can say how fast the model is, what it costs, and how to serve an answer without making the user wait.
- 23 min
Lesson
How an LLM looks to a system designer: tokens, speed, cost, context limit
What a token is, why you pay for input and output tokens at different prices, why output length decides most of the wait, what the context window limits, and how to work out the cost of an AI feature for 1,000 users.
- 24 min
Lesson
Serving an AI feature: streaming, timeouts, fallbacks and limits
How to show an AI answer fast (streaming), which three time limits to set, what to do when the model service fails (a second model, then an honest error), and how to live inside the provider's limits with a waiting line.
- 12 min
Trade-off
Synchronous vs asynchronous calls
When a caller should wait for the answer and when it should hand work off, the numbers that decide, and what breaks with each choice.
- 13 min
Trade-off
Retry vs fail fast
When to retry a failed call and when to fail fast and return an error or a fallback, the numbers that decide, and what breaks with each.
- 7 min
DrillFree account
A payment status service is flooded on salary day
Practise this decision: retry the call or fail fast. Seven minutes.
- 22 min
Lesson
RAG: letting the model read your documents (embeddings, vector search, chunking, freshness)
How an app lets a model answer from your own documents: embeddings, finding the nearest chunks, the HNSW idea, chunk size and overlap, metadata filters, re-indexing, and when keyword search or the whole text in the prompt is enough.
Stage 3 of 5 · 2 h
Faster, cheaper and safer
You can cache AI answers, check that they are good and safe, and keep the bill under control.
- 12 min
Lesson
Caching for AI: prompt cache and meaning cache
Two ways to pay less for AI calls: the provider's prompt cache for the unchanging start of a prompt, and your own meaning cache that reuses an old answer for a similar question, with the rules that keep both safe.
- 15 min
SimulatorFree account
Simulator Level 3: add a cache
Add a cache and see how hit ratio and TTL change the load. The same ideas apply to AI answers.
- 24 min
Lesson
Is the AI answer good and safe? Evals, prompt injection and output checks
How to test an AI feature with a fixed set of cases scored by code, a second model and people, what prompt injection is, and four defences written in code that still work when the model is fooled.
- 12 min
Lesson
Keeping the AI bill under control: routing, budgets and per-user limits
Five ways to keep an AI bill small and safe: send easy work to a small model, cap output length, batch work that can wait, give each user a daily budget with a token bucket, and alert on spend.
- 15 min
Cost lab
AI chatbot feature (LLM token costs)
Build the bill for an AI chatbot and bring it under the budget.
- 15 min
Trade-off
Backpressure queue vs drop vs degrade
How to choose between queueing, dropping and degrading when work arrives faster than a service can finish it, with queue sizes worked out.
- 7 min
DrillFree account
The site freezes in the sale and users give up
Practise this decision: queue, drop or do less when too much arrives. Seven minutes.
- 13 min
Trade-off
Token bucket vs sliding window rate limiting
Which rate limiting method to choose: token bucket or sliding window. The numbers that decide, the bursts each one allows, and what breaks.
- 7 min
DrillFree account
A rate limit must allow the burst of an app screen
Practise this decision: token bucket or sliding window. Seven minutes.
Stage 4 of 5 · 52 min
Agents and a full example
You can design an agent that uses tools safely, and you have seen a whole AI product designed.
- 24 min
Lesson
AI agents and tools: loops, safety and idempotent tools
What an AI agent is (a model that calls tools in a loop), when a fixed workflow is better, and the controls that keep an agent safe: limits on steps, time and money, least privilege, approval, idempotent tools, a log of each step, and care with tool results.
- 28 min
Lesson
Case study: AI interview coach (Tradeoff itself)
A real AI feature, taken apart: the server keeps the rules and the model only proposes, the model's JSON is checked, every quote in a report is checked, the fixed prompt start is cached, left sessions are finished lazily, and the cost and fair-use limits are worked out.
Stage 5 of 5 · 1 h 37 min
Practise
You have designed an AI system under a clock, and answered a twist from the interviewer.
- 7 min
- 45 min
Design mockFree account
Design an AI Chat Assistant for a Product
A mock about streaming, cost per answer and a model that is down. Seven more AI problems wait in the design list.
- 45 min
Design mockCheckpointFree account
Design an AI Agent That Takes Multi-Step Actions
Your checkpoint: a hard mock about an agent that takes real actions, with a twist from the interviewer.
Estimates
How the time is counted
These times are planning estimates, not deadlines. Yours will vary.
- Lessons and trade-off pages: the number of words divided by 130, rounded, plus 4 minutes to think.
- A Drill: about 7 minutes. A system design mock: allow about 45 minutes. A machine coding mock: allow about 75 minutes.
- Other mocks: a SQL mock with one problem, about 29 minutes. A behavioural mock, about 36 minutes. A resume mock, about 35 minutes. A coding mock, about 45 minutes.
- A simulator level or a Cost Lab workload: allow about 15 minutes. This is our own estimate for a first try. It can take you less or more.
- Incident investigation: time varies. The room has a simulated clock; your real time depends on how you investigate. It has no fixed time, so it is not counted in any total.
- The total is the sum of all steps that have a time. The weeks are the hours divided by 4, rounded up. We assume 4 hours a week, like a person with a job. Doing a step again is not counted.
Ready for the checkpoint?
The last step is a mock interview with an AI interviewer. It gives you a report that shows what you did well and what was missing.