There is a peculiar kind of architectural decision that arrives in the form of a mandate. A new platform team is formed, a vendor is selected, and three quarters later the slack channel is full of people asking why their staging cluster is throwing 502s during what used to be ordinary deploys. This is the service mesh story, and it is not unique to any one team.

Over the last eighteen months I've sat through retros at six different companies who adopted a mesh — Istio, Linkerd, App Mesh, take your pick — and watched the same shape of problem emerge. The pitch is always identical. The outcome rarely is.

The pitch is rarely the problem

In every case, the mesh was sold as solving observability, mutual TLS, and traffic shaping in one package. And to be clear: a mesh can absolutely do those things. The question is whether you have the problem the mesh is solving, or whether you have a different problem that the mesh will now sit on top of.

1apiVersion: networking.istio.io/v1beta1
2kind: VirtualService
3metadata:
4  name: checkout
5spec:
6  hosts: ["checkout"]
7  http:
8    - route:
9        - destination: { host: checkout, subset: v1 }
10          weight: 90
11        - destination: { host: checkout, subset: v2 }
12          weight: 10

A canary routing block like the one above looks innocuous. It is not. It is now a piece of production infrastructure that lives in YAML, is applied by a controller, is enforced by a sidecar, and is invisible to the application code that depends on it being correct.

What actually breaks

The failures cluster into three categories. First: latency. The sidecar adds a millisecond, sometimes three. For most services this does not matter. For the one service that fans out to forty downstream calls in a single request, it matters a great deal. Second: configuration drift, where the team running the mesh and the team running the service stop agreeing about what counts as a sane default. Third — and this is the killer — debugging.

We spent two weeks chasing a bug that turned out to be the sidecar buffering a connection during a rolling restart. Two weeks. The bug report said "checkout is slow on Tuesdays".

There is a specific kind of fatigue that sets in when an engineer has to reason about three layers of network stack to explain a single 504. The mesh promised to make the network observable; for the median on-call engineer, it made the network stranger.

What I'd do instead, mostly

For the teams I've advised who were already mid-adoption, the answer is rarely to rip it out. The answer is to be honest about what it's for. If you wanted mTLS, you have mTLS — stop using the mesh for traffic shaping. If you wanted golden signals, install OpenTelemetry on the services themselves and let the mesh be a transport detail. The mesh is at its best when it is small and boring.

For greenfield teams: do not start with a mesh. Start with library-level instrumentation, a load balancer you understand, and certificate rotation that is one cron job away from being legible. You can add a mesh when you have the failure modes that justify it.