Keeping the AI bill under control: routing, budgets and per-user limits

A free AI feature goes live, and the bill is ten times the plan. How do you control it?

A weak answer is "use a cheaper model". A strong answer sizes the model to each request, caps the output, and limits each user and the total.

Think of a phone plan: each user gets a data pack, so one heavy user cannot use all the data.

Lesson 118 gave the formula: cost of one call = input tokens x input price + output tokens x output price. The cheapest call is the one you do not make: reuse a saved answer, or pay less for a prompt start that the provider has seen before (lesson 121). Five controls follow:

1. Route by difficulty. Routing sends each request to the model that fits it. Anthropic's pricing page (20 Oct 2026) lists a large model at $4 and a small model at $0.10 per million input tokens: 40 times less. A classifier is a cheap first call that labels a request easy or hard; easy ones go to the small model. In cheap first, the small model answers and a check decides if it is good enough. If not, the large model tries. A classifier adds a call to every request and can label wrongly. Cheap first pays twice for a hard request.
2. Cap the output. Set a maximum for output tokens. The model stops there.
3. Batch work that can wait. A batch is many calls sent together to be answered later, such as a nightly summary. Anthropic charges 50% less. A batch finishes within 24 hours or expires.
4. Budget for each user and day. The provider limits the whole account, not each user. Give each user a token bucket (lesson 56) that holds rupees. Before a call, take the worst case (input cost plus capped output cost); after it, give back what was not used. Add a total daily cap, because each fake account gets its own free budget.
5. Alert on spend. Add up the cost of every call and alert at 80% of the daily cap. Set the provider's monthly limit; at that limit, calls fail for everyone.

SwadKart has 1,000 chats a day. Assumptions: prices as in lesson 118. The small model costs one tenth (Rs 10 and Rs 50 per million tokens). 70% of chats are easy. The classifier runs once for each chat on the small model, with 300 input and 5 output tokens. The cheap-first check is free code.

• No routing: 1,000 x Rs 1.10 = Rs 1,100.
• Classifier: its call costs 300 x Rs 10 / 1,000,000 + 5 x Rs 50 / 1,000,000 = Rs 0.00325. A chat costs 0.7 x 0.11 + 0.3 x 1.10 + 0.00325 = Rs 0.41 (the predict box: about Rs 0.40). A day is Rs 410.25, a 63% saving.
• Cheap first: a hard chat costs 0.11 + 1.10 = Rs 1.21. Average: 0.7 x 0.11 + 0.3 x 1.21 = Rs 0.44.
• Cap: 4,000 output tokens cost 4,000 x Rs 500 / 1,000,000 = Rs 2.00.
• Batch: 1,000 summaries (8,000 input and 200 output tokens each, large model) cost 0.80 + 0.10 = Rs 0.90 each, so Rs 900. Batch price: Rs 450.
• Budget: B = Rs 2, r = Rs 2 a day. A user spends at most B + r x T = 2 + 2 x 1 = Rs 4 on day one, then Rs 2 a day. 1,000 fake accounts spend Rs 4,000, then Rs 2,000 a day. Normal use is Rs 410, so set a total cap of Rs 800 and alert at 80%: Rs 640.

• Routing solves most of the bill (Rs 1,100 to Rs 410). It costs wrong labels. I would test it on fixed cases (lesson 122).
• A cap solves the worst case. It costs cut answers. I would set it from the p95 length (lesson 4).
• A budget for each user solves one user taking all. It costs angry users, and misses many accounts. I would add a total cap.

"The provider's spend limit is my budget." It covers the whole account, so one user can stop everyone.

"A limit for each user is enough." 1,000 free accounts at Rs 2 each cost Rs 2,000 a day.

1. Name the five controls. Why is the provider's limit not enough?
2. A day has 2,000 chats, 60% easy. With the prices of the predict box, what does a day cost with and without routing?
3. A script makes 5,000 free accounts, each with a Rs 2 bucket refilled Rs 2 a day. What is the most spent on day one, and each day after? What do you add?