Service mesh vs client libraries vs nothing

The decision

Services that call each other need the same safe habits: timeouts, retries, encrypted traffic, load balancing and metrics. You have three places to put them. Nothing means each service sets these in its own code with its own settings. A client library is one shared package that every service imports. A service mesh puts a small proxy next to each service. The proxy, called a sidecar, handles the habits outside your code, and a control plane sends it the rules. The forces are cost, how fast you can change a rule, how many languages you use, and how much the team must learn.

Choose nothing when

• You have fewer than about 10 services and a small team that talks daily (Assumption: our rough limit).
• The platform you already use gives load balancing and encryption, such as a managed load balancer.

Choose a client library when

• Most services use one or two languages.
• You need shared behaviour: timeouts, a retry limit, a circuit breaker, headers for tracing.
• You can ask teams to upgrade within weeks, and a slow upgrade is acceptable.
• You want no extra servers and no extra network hop.

Choose a service mesh when

• You use many languages, so one library per language is a burden.
• You must encrypt all service traffic with mutual authentication, and show an auditor the proof.
• You need to change a rule for all services in minutes, for example during an incident.
• A platform team exists that can run the control plane and the upgrades.
• You can start with the lighter mode of a mesh, which has no sidecar in each pod (see the numbers).

The numbers that change the answer

Cost of sidecars (cited fact plus calculation). The Istio project measured one sidecar. The test used 2 worker threads, 1,000 requests a second and 1 KB payloads. The sidecar used about 0.20 vCPU and 60 MB of memory (Istio 1.24, see Sources). Assumption: 40 services with 300 pods in total, and each pod near 1,000 requests a second. A lighter load costs less.

• CPU: 300 x 0.20 = 60 vCPU.
• Memory: 300 x 60 MB = 18,000 MB = 18 GB.
• Assumption: 1 vCPU costs Rs 3,000 a month and 1 GB costs Rs 400 a month. Cost = 60 x 3,000 + 18 x 400 = 180,000 + 7,200 = Rs 187,200 a month, or Rs 2,246,400 a year.
• Add one platform engineer for the mesh: Rs 3,000,000 a year (Assumption). Total: Rs 5,246,400 a year.

Cost of a library (assumptions plus calculation). An engineer-week costs 3,000,000 / 48 = Rs 62,500.

• Build three libraries (three languages) at 6 weeks each: 18 weeks = Rs 1,125,000, once.
• Upkeep: 3 x 2 = 6 weeks a year = Rs 375,000.
• Each fix must be adopted by every service. Assumption: 2 engineer-hours a service, 40 services, 6 fixes a year. That is 6 x 40 x 2 = 480 hours = 12 weeks = Rs 750,000.
• Total a year after the build: 375,000 + 750,000 = Rs 1,125,000.

What the numbers say. On cost alone the mesh does not win here: Rs 5.25 million against Rs 1.13 million a year. The mesh pays back on things the cost lines do not show. A library fix takes 2 engineer-weeks of work (480 hours / 6 fixes / 40 hours a week). Assumption: it takes 3 calendar weeks to reach all 40 services. A mesh rule changes for all services in minutes. The gap is 5,246,400 - 1,125,000 = Rs 4,121,400 a year. Choose the mesh when speed, mutual encryption for audits, or many languages are worth that gap.

Latency (assumption plus calculation). Assumption: each proxy adds 2 ms, and a call crosses two proxies. That is 4 ms a call. A request with 5 calls in a row adds 5 x 4 = 20 ms. The Istio page shows latency in charts only, so measure your own.

A lighter start (cited fact). Istio's ambient mode removes sidecars from pods. A per-node component handles layer 4 duties such as mutual encryption and basic authorization, and optional waypoint proxies handle layer 7 duties such as routing. You can adopt it step by step (see Sources).

What breaks with each choice

Nothing.

• Settings differ, and one service has no timeout. A slow dependency then holds all its threads. Fallback: a checklist in code review and a test that fails when a client has no timeout.

Client library.

• One language has an old version with a bug, and nobody knows which services still use it. Fallback: report the library version from every service and set an upgrade date.
• A retry bug in the library multiplies load in an incident. Fallback: a retry budget, such as at most 10% extra calls, in the library.

Service mesh.

• A wrong rule pushed to all services breaks every call at once. Fallback: roll out rules to one namespace first, and keep the previous version to restore.
• The sidecar adds CPU, memory and latency, and it can fail on its own. Fallback: alert on proxy errors and keep resource limits tuned.
• The control plane is down during a change. Fallback: check, in a test cluster, that proxies keep working with the last rules they received.

What it costs

• Money: about Rs 5.25 million a year for the mesh and Rs 1.13 million for libraries in the example, both assumptions. Nothing costs no money until the first incident.
• Reversible-first path. Start with nothing and a one-page standard for timeouts. Build a thin library when you reach about 10 services, with timeouts, retry budget and trace headers only. Move to a mesh when one of three triggers is crossed (our assumptions). The triggers are more than 3 languages, a mutual-encryption requirement from an auditor or customer, or a need to change rules everywhere in under 3 weeks. Begin with the ambient layer 4 mode.
• Changing your mind: library to mesh is hard to do twice if both retry, because retries multiply. Turn off the library's retries in a service before the mesh retries for it. Assumption: plan a quarter for 40 services.

How each level answers

The question is: "40 services call each other. Do we need a service mesh?"

SDE-1: A service mesh puts a proxy next to every service to handle retries and encryption. A library does it inside the code.

SDE-2: I use a shared library for timeouts and retries if we use one language. A mesh helps when there are many languages.

Senior: A sidecar costs about 0.20 vCPU and 60 MB at 1,000 requests a second. So 300 pods cost 60 vCPU and 18 GB, about Rs 2.2 million a year. A library costs less but takes about 3 weeks to roll out a fix. I would begin with a library and a retry budget. I move to a mesh when we pass 3 languages or an audit asks for mutual encryption. I watch for double retries.

Staff: I frame it as the speed of changing a rule against the cost of owning a platform. In the example the gap is about Rs 4.1 million a year. I would not start a mesh without a platform team that owns it and its on-call. I measure the time to roll out a fix, services without a timeout, and proxy errors. The path is a standard, then a library, then the lighter mesh mode. I revisit it every year, or when a new language or an audit arrives.

Say it in the interview

With 40 services in three languages, I would start with one shared library for timeouts, a retry budget and trace headers. A sidecar costs about 0.20 vCPU and 60 MB for each pod at 1,000 requests a second, so a mesh needs a clear reason. I would move to a mesh for audited mutual encryption or when a fix must reach every service in minutes instead of weeks.

Common mistakes

• Adding a mesh to look modern. It is a platform you must run. Without a team for it, it becomes the next incident.
• Retries in both the library and the mesh. With 3 tries at each layer, one call becomes up to 9. Retry in one place.
• Skipping the version report for a library. Without it, you do not know who runs the buggy version.

Sources

• Istio, performance and scalability (Istio 1.24): one sidecar with 2 worker threads at 1,000 HTTP requests a second with 1 KB payloads uses about 0.20 vCPU and 60 MB of memory; latency is shown in charts, not in text. https://istio.io/latest/docs/ops/deployment/performance-and-scalability/ (opens in a new tab) (checked 26 Oct 2026)
• Istio, ambient mode overview: the ztunnel handles layer 3 and 4 duties such as mutual TLS; optional waypoint proxies run outside application pods and handle layer 7 duties; adoption can be step by step. https://istio.io/latest/docs/ambient/overview/ (opens in a new tab) (checked 26 Oct 2026)
• Pod counts, prices, salaries, weeks to build, hours per fix, number of fixes, proxy delay and the trigger values are our assumptions. The Rs 5.25 million, Rs 1.13 million and Rs 4.1 million are our calculations, shown above.