Sidecar, ambassador and service mesh

A sidecar is a helper process that runs beside each instance of your service and shares its lifecycle. An ambassador is a sidecar that proxies the service’s outbound calls. A service mesh puts a proxy beside every service and manages all of them from one control plane. Together they move the plumbing every microservice needs (mTLS, retries, timeouts, load balancing, telemetry) out of application code and into infrastructure, at the price of CPU, memory, latency and one more system to run.

Picture it first: an assistant for every manager

Picture a big company where managers phone each other all day. Every call needs the same routine: find the colleague’s current extension, redial if busy, give up after a minute, prove who you are, and log the call for the auditors.

You could train all 60 managers in the routine, but each trainer teaches it a little differently, and a rule change means months of retraining. Or you could seat an assistant beside every manager. The manager says “get me Payments” and the assistant does the rest, while head office sends every assistant the same directory and a fresh ID badge each morning. An old-school manager who won’t learn the new phones gets an assistant who places all their outgoing calls. A cheaper variant is one reception desk per floor, plus a specialist desk for complicated requests.

Map it back. Training every manager is the in-app library. The assistant is a sidecar; placing calls for the old manager is an ambassador; assistants plus head office are a service mesh and its control plane. The floor’s reception desk is a node proxy (Istio’s ztunnel) and the specialist desk is a waypoint.

The problem: the same plumbing in every service

Once you split a system into services (monolith vs microservices), every network call needs the same cross-cutting code:

• Discovery and load balancing: find healthy instances and spread calls across them (service discovery, L4 vs L7 load balancing).
• Timeouts, retries and circuit breaking, so one slow dependency can’t hang its callers (timeouts and retries, circuit breaker).
• mTLS and identity: encrypt every call and prove which service is calling (service-to-service auth and mTLS).
• Telemetry and policy: rate, errors and latency per route, trace spans, and who may call whom (observability).

The running example is a Swiggy-style food-delivery platform: 60 services in Go, Java and Node, running as 800 pods. Its security team wants every call in the payments scope encrypted and authenticated before a PCI DSS audit, and its teams want to canary every release.

The fat client library

The first answer is a shared library, the approach Netflix popularised with its open-source Java libraries for discovery, client-side load balancing and circuit breaking. It works well in one language. This platform has three: 8 concerns × 3 languages = 24 implementations that should behave identically and don’t. The Go library retries on a 503, the Java one doesn’t, and the Node one is two versions behind.

Worse, the library ships inside every binary. Rotating a TLS setting or fixing a retry bug means 60 teams rebuild and redeploy, and the fleet is only as current as its slowest team. That coupling is the main reason to move plumbing out of process.

The library keeps real advantages: no extra hop, no extra container, and knowledge of the app’s context, such as which calls are safe to retry. When only one language needs the feature, Azure’s guidance prefers it.

The sidecar pattern

A sidecar is a separate process or container deployed alongside each instance of an application and created and retired with it, like a motorcycle sidecar. Azure’s Sidecar pattern notes it is also called the sidekick pattern. In Kubernetes its home is the pod: containers in a pod share a network namespace, so the app reaches its sidecar on localhost, and they can share volumes.

Common sidecars include log shippers that tail a shared volume, secrets agents that fetch and rotate credentials, runtimes such as Dapr that offer state and pub/sub through a local API, and proxies that handle the app’s traffic: the building block of a mesh.

What you gain: one implementation for every language, updates independent of the app, a separate resource limit, and ownership by a platform team. What you pay: latency on every interaction, and a per-instance resource cost that Azure warns can outweigh the benefit for small applications. One sidecar is cheap; 800 pods means 800 of them.

Start-up and shutdown ordering

A classic sidecar was just a second container, and Kubernetes didn’t guarantee it was ready before the app started or that it stopped after the app. Three bugs followed:

1. The app starts before its proxy. Traffic is redirected to a proxy that isn’t listening yet, so the first calls fail and the app may crash-loop.
2. The proxy exits first on shutdown. In-flight requests fail while the app is still draining.
3. Jobs never finish. The batch container exits, but the sidecar keeps running, so the Job’s pod never completes.

Kubernetes native sidecar containers fix all three. You declare the sidecar as an init container with restartPolicy: Always. It starts before the app containers (after its startup probe passes, if it has one), runs for the pod’s whole life, is stopped only after the app containers exit, and doesn’t block a Job from completing. The feature first shipped in v1.28, has been on by default since v1.29 and is stable since v1.33. Linkerd has injected its proxy this way by default since version 2.20.

spec:
  initContainers:
    - name: proxy
      image: example/proxy:1.0
      restartPolicy: Always      # makes this a native sidecar
      startupProbe:
        httpGet: { path: /ready, port: 8081 }
  containers:
    - name: orders
      image: example/orders:4.2

The ambassador and the adapter

An ambassador is an out-of-process proxy, co-located with the client, that sends network requests on the client’s behalf (Azure’s Ambassador pattern). The app calls a local address; the ambassador finds the remote service, checks its breaker, adds TLS and tracing headers, retries if allowed and forwards the call. To the app, it represents the remote service.

Its sweet spot is code you can’t change. Say the platform’s settlement system is a ten-year-old Java app calling a bank’s payout API over plain HTTP with no timeouts. Rewriting it takes a quarter; an ambassador that adds mTLS, a 2-second timeout and a breaker takes days, and the platform team can own it.

Azure’s caveats are worth repeating:

• Retry only idempotent operations. The proxy can’t know that a payout POST must never be repeated. Let the client opt out with a header, or use an idempotency key.
• It adds latency, so with only one language a library may be simpler.
• Per instance or shared: a sidecar per instance, or one daemon for every process on a host.

An ambassador is usually deployed as a sidecar, so the words answer different questions: sidecar is where it runs, ambassador is what it does. A third member of the family, the adapter, works the other way round: it presents the app to the platform in a standard shape, for example by turning a legacy app’s custom stats into a Prometheus /metrics endpoint.

Service mesh: a proxy beside every service

A service mesh applies the sidecar to every service and adds a control plane to manage the proxies. In Istio, the data plane is a set of Envoy proxies that carry every byte between services and report telemetry; the control plane, istiod, configures them from your rules and the current state of the cluster.

The data plane: transparent interception

The app still calls http://payments:8080 and never knows a proxy exists. When the pod starts, an init container (or a CNI plugin) installs iptables rules in the pod’s network namespace that send all inbound and outbound TCP traffic through the local proxy. So every call crosses two proxies:

• The outbound proxy, in the caller’s pod, does discovery, load balancing, timeouts, retries and circuit breaking, and starts mTLS.
• The inbound proxy, in the callee’s pod, terminates mTLS, enforces authorization policy and hands the request to the app on localhost.

Both report metrics, so you get request rate, errors and latency for every pair of services without touching code. Linkerd documents the same outbound and inbound split.

The control plane: config and identity

istiod does three jobs:

1. Discovery. It watches the Kubernetes API for services, pods and endpoints.
2. Configuration. It turns your high-level rules (routes, retries, policies) into Envoy configuration and streams it to every proxy over the xDS APIs. How proxies acknowledge or reject each version, and keep serving if istiod is down, is covered in control plane vs data plane.
3. Certificates. istiod is a certificate authority. In each pod, the Istio agent creates a private key and a signing request, istiod signs it, and the agent hands the certificate to Envoy, repeating the process before expiry so rotation is automatic.

Each certificate carries the workload’s identity as a SPIFFE ID derived from its Kubernetes service account, such as spiffe://cluster.local/ns/payments/sa/checkout. Authorization policies use these identities (“only orders may call payments”) instead of pod IPs, which change on every restart.

What the mesh gives you

With a programmable proxy on every call, configuration changes behaviour fleet-wide:

• mTLS everywhere, with certificates rotated automatically. For the PCI scope, one STRICT policy on the payments namespace refuses every plaintext call.
• Traffic shifting. Send 5% of calls to payments v2 and 95% to v1, or send only requests carrying an internal tester header to v2. Rolling back is a config change, not a redeploy (canary and blue-green).
• Resilience: per-route timeouts, retries, connection limits and outlier detection, which ejects an instance that keeps failing from the load-balancing pool.
• Fault injection: delay 1 in 1,000 calls by 5 seconds, or return 503s, to prove your timeouts and fallbacks work.
• Uniform telemetry and authorization: the same golden metrics for Go, Java and Node, and policies from L4 (“orders may call payments”) to L7 (“only GET on /menu”).

A canary is a few lines of YAML. A DestinationRule, not shown, defines subsets v1 and v2 by pod label:

apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: payments
spec:
  hosts: [payments]
  http:
    - match:
        - headers:
            x-canary: { exact: "yes" }
      route:
        - destination: { host: payments, subset: v2 }
    - route:
        - destination: { host: payments, subset: v1 }
          weight: 95
        - destination: { host: payments, subset: v2 }
          weight: 5
      timeout: 2s

Two Istio defaults surprise people: the HTTP timeout is disabled by default, so set timeouts explicitly, and Istio retries HTTP requests twice by default, which matters below.

Linkerd is the lighter alternative with the same shape. Its control plane has a destination service (discovery and policy), an identity service (the certificate authority) and a proxy injector. Its data plane is an ultralight micro-proxy written in Rust, built only for the mesh rather than as a general-purpose proxy like Envoy. It turns on mTLS automatically and load-balances by latency. You trade some of Istio’s L7 features and extensibility for less to configure and run.

What a mesh costs, in numbers

Istio publishes measurements for its 1.24 release, at 1,000 HTTP requests per second of 1 KB each with 2 worker threads:

Memory also grows with the configuration a proxy holds (listeners, clusters and routes).

Fleet arithmetic. Suppose the food platform makes 50,000 service-to-service calls a second at dinner peak. Each call crosses two sidecars, so the proxies handle 100,000 requests a second. Scaling Istio’s figure linearly (a rough approximation), that is 100 × 0.20 = 20 vCPU of proxy work. Memory depends mostly on config, not traffic: 800 sidecars × 60 MB ≈ 48 GB.

What you reserve is usually more, because each sidecar is sized for its own pod’s peak; Istio’s comparison calls this wasteful. At an illustrative request of 0.1 vCPU and 128 MiB per sidecar, 800 pods reserve 800 × 0.1 = 80 vCPU and 800 × 128 MiB = 100 GiB: 20% on top of apps that request 400 vCPU.

Latency. In the same benchmark (bare metal, mTLS on, 1 KB requests at 1,000 per second), p99 latency was 0.18 ms with no mesh and 0.72 ms with sidecars: about 0.54 ms added per call by the two proxies. A checkout that makes 4 sequential calls adds about 4 × 0.54 ≈ 2.2 ms, roughly 1% of a 200 ms budget. Expect more with bigger payloads, busy nodes and telemetry filters, which Istio says have a moderate impact; measure your own.

Operations cost more than the CPU. A new proxy version reaches a pod only when the pod restarts, so every mesh upgrade rolls every workload; Istio’s maintainers list this among the main complaints about sidecars. A failed call might be the app, either proxy, iptables or a policy, so engineers must learn to read proxy logs and config dumps. And one bad route reaches hundreds of proxies within seconds.

The retry amplification trap

Retries multiply across layers. Suppose a request goes gateway → orders → payments → bank API, and each layer makes up to 3 attempts. When the bank API slows down, the gateway tries orders 3 times, each of those tries payments 3 times, and each of those tries the bank 3 times: 3 × 3 × 3 = 27 attempts for one tap in the app, arriving exactly when the bank can least cope (retry storm).

A mesh makes this easy to fall into. If each app makes 3 attempts and its outbound proxy makes 3 attempts per call, every hop makes 3 × 3 = 9 attempts, and three hops make 9³ = 729. Because Istio retries twice by default, a team that already retries in code is stacking two layers without knowing it.

The fix:

• Retry in one layer only. With a mesh, that usually means deleting retry loops from apps and setting retries per route in the mesh, or turning mesh retries off where the app owns them.
• Cap the total: a retry budget (retries at most, say, 10% of requests), per-try timeouts that fit inside an overall deadline, and backoff with jitter.
• Retry only idempotent calls. Never auto-retry a payout POST without an idempotency key.

Sidecarless: ambient, eBPF and proxyless gRPC

A proxy per pod, restarts to upgrade and ordering bugs pushed meshes towards designs with no proxy inside the pod.

Istio ambient mode

Ambient mode splits the mesh into two layers:

• ztunnel, one per node (a DaemonSet), handles L4: mTLS, L4 authorization and TCP telemetry. It doesn’t parse HTTP. Node-to-node traffic travels in an mTLS tunnel built on HTTP CONNECT, which Istio calls HBONE.
• Waypoint proxies are ordinary Envoy deployments outside your pods. They add L7 features (HTTP routing, retries, L7 authorization, request metrics) only for the namespaces or services that need them, and scale like any other deployment.

Istio declared ambient mode generally available in 1.24, on 7 November 2024. A namespace joins the mesh when you label it, with no pod restarts, and mesh upgrades no longer touch application pods. Security changes too: with sidecars, a compromised app pod exposes that pod’s mesh keys; with ambient, the pod holds none, and each ztunnel holds keys only for workloads on its node.

In the benchmark above, ztunnel alone added about 0.31 − 0.18 = 0.13 ms at p99 and a waypoint about 0.49 ms, since L7 work happens once, at the waypoint, not in two sidecars. For the food platform on, say, 50 nodes: 50 ztunnels × 12 MB = 0.6 GB against 48 GB of sidecars, and 100,000 ztunnel traversals a second cost about 100 × 0.06 = 6 vCPU. A payments waypoint at 10,000 calls a second adds 10 × 0.25 = 2.5 vCPU. If every namespace needs L7 the CPU gap narrows; memory and reservations are where ambient saves most.

The price: L7 features exist only where you deploy a waypoint, L4-only traffic gets no per-request metrics or retries, and Istio’s comparison page still lists multi-cluster, multi-network and VM support as not yet implemented in ambient mode.

eBPF-based meshes

Cilium takes the idea into the kernel: eBPF handles IP, TCP and UDP in the Linux datapath, and an Envoy proxy parses HTTP, gRPC and DNS only where L7 is needed. As with ambient mode, L4 is cheap everywhere and you pay for L7 only where you use it.

Proxyless gRPC

If your services speak gRPC, the library itself can be the data plane. gRPC’s design proposal A27 added an xDS client to gRPC, implemented in C-core, Java and Go. The client reads a bootstrap file named by GRPC_XDS_BOOTSTRAP, connects to an xDS server such as istiod, and a channel to xds:///payments:8080 takes its endpoints, load-balancing policy and routes from the control plane. No proxy, no extra hop; Istio lists proxyless as a third data-plane mode. The catch: you are back to a library, covering only gRPC traffic in languages whose gRPC supports the xDS features you need, and upgraded by rebuilding the app.

When you need a mesh, and when you don’t

A mesh starts to pay off, as a rule of thumb, with roughly 20 or more services owned by several teams, two or more languages, a hard requirement for mTLS everywhere, frequent canaries or traffic shifting, and a platform team that can run the mesh like any other production system.

You probably don’t need one for a handful of services in one language. A good shared library, a load balancer and an API gateway give you most of the benefit at a fraction of the cost.

Mesh vs API gateway. A gateway sits at the edge and handles north–south traffic: external clients, end-user authentication, API keys, per-customer rate limits and public TLS. A mesh handles east–west traffic between your services: workload identity, mTLS and per-route resilience. Most platforms run both, and meshes often ship an ingress gateway on the same proxy (proxy vs reverse proxy).

The food platform clears the bar. A sensible plan: ambient L4 in every namespace for cheap mTLS, waypoints where canaries and L7 policy matter, and the edge left to the API gateway.

Going deeper: failure modes at scale

Proxy not ready at start-up. Without native sidecars, the app’s first calls hit a proxy that isn’t listening. Use native sidecars or hold the app until the proxy is ready.

Sidecar OOM kills. In an unscoped mesh every proxy learns every endpoint in the cluster, so memory creeps up as the cluster grows until the kernel kills a proxy and its pod loses all traffic. Scope each proxy’s config to the services it actually calls (Istio’s Sidecar resource does this) and alert on proxy memory.

A bad config pushed fleet-wide. Envoy rejects config it can’t apply, but a valid, wrong rule (100% of traffic to a broken v2, a policy that denies orders) is accepted by every proxy within seconds. Treat mesh config like code: review it, validate it in CI, roll it out namespace by namespace, and keep rollback one revert away (control plane vs data plane).

PERMISSIVE to STRICT. Istio’s mesh-wide default is PERMISSIVE: workloads accept both mTLS and plaintext, so services can join gradually. Switch a namespace to STRICT before every caller has a proxy and you break callers such as a cron job outside the mesh. Use mesh telemetry to find remaining plaintext connections, then tighten namespace by namespace; your auditor will want STRICT.

Stateful and headless workloads. Databases and brokers use headless services, pod-to-pod addressing and long-lived connections, and server-first protocols need extra configuration in sidecar mode. Many teams keep data stores out of the mesh or at L4 only.

Egress. For card data, send traffic leaving the cluster through a dedicated egress gateway with an allow-list of partner hosts, and alert on anything else.

xDS push storms. Istio says istiod’s CPU grows with the rate of deployments and config changes and the number of connected proxies. A rolling deploy of a 40-pod service removes 40 endpoints and adds 40: 80 changes. If all 800 proxies watch that service, that is up to 80 × 800 = 64,000 proxy updates; scoped to the 100 proxies that call it, 8,000. istiod scales horizontally, but scoping is the bigger lever, and Istio recommends it at scale.

Cheat sheet

How to say it in the interview

“Every service needs discovery, load balancing, timeouts, retries, mTLS and telemetry. With 60 services in Go, Java and Node, a library per language drifts and ties every fix to 60 releases, so I’d move that plumbing out of process: a proxy on every call path, managed by a control plane that pushes routes and issues short-lived certificates per service identity. That gives me STRICT mTLS for the PCI scope, 95/5 canaries by config and the same metrics everywhere. Sidecars cost about 60 MB each and half a millisecond per call, so I’d start with ambient mode for cheap L4 mTLS and add waypoints only where we canary. And I’d retry in one layer only.”

Sidecar or library? A library has no extra hop and knows the app’s context, so it wins with one language and a few services. A sidecar wins with several languages or code you can’t change, at the price of a proxy per pod and some latency.

How is a mesh different from an API gateway? A gateway handles north–south traffic at the edge: external clients, end-user auth, API keys. A mesh handles east–west traffic between services: workload identity, mTLS, retries, traffic shifting. Most platforms use both.

What does a mesh cost? In Istio’s benchmark, about 0.2 vCPU per 1,000 requests a second and 60 MB per sidecar, plus about 0.5 ms at p99 per call. For 800 pods that is about 48 GB of memory, plus restarts to upgrade, harder debugging and fleet-wide config risk.

Does the mesh give you tracing for free? Metrics, yes. Traces, only partly: proxies emit spans, but each app must forward the trace headers from incoming requests to its outgoing calls, or the spans never join up.

What is ambient mode? Istio’s sidecarless data plane, GA since 1.24: a ztunnel per node does mTLS and L4 policy, and optional waypoint proxies add L7 features per namespace. No proxy in the pod, no restarts to join or upgrade and far less memory, but L7 only where you deploy a waypoint.

Where to go next

• Monolith vs microservices: where the plumbing problem comes from.
• Control plane vs data plane: xDS, ACK and NACK, and safe config rollouts.
• Service-to-service auth and mTLS: certificates, identity and authorization.
• API gateway: the north–south counterpart to a mesh.
• Timeouts, retries and backoff with jitter and retry storm: retries that help instead of hurt.
• Deployment strategies: the canaries a mesh can drive.
• Observability: golden metrics and trace propagation.

References and further reading

1. Sidecar pattern (Azure Architecture Center)learn.microsoft.comDefinition and shared life cycle, language independence, the per-instance resource and latency costs, native Kubernetes sidecars, and when not to use the pattern.
2. Ambassador pattern (Azure Architecture Center)learn.microsoft.comAn out-of-process proxy co-located with the client, its use for legacy apps, retries only for idempotent calls, client opt-out headers, and sidecar versus shared deployment.
3. Kubernetes documentation: Sidecar Containerskubernetes.ioInit containers with restartPolicy: Always, start-up and termination ordering, Jobs, shared network and volumes; first in v1.28, on by default from v1.29, stable since v1.33.
4. Istio documentation: Security conceptsistio.ioistiod as certificate authority, the agent’s CSR and SDS flow with automatic rotation, PERMISSIVE and STRICT peer authentication, and the PERMISSIVE mesh-wide default.
5. Istio documentation: Traffic management conceptsistio.ioPer-service timeouts (HTTP timeouts disabled by default), retries (twice by default), circuit breakers and fault injection.
6. Istio documentation: Performance and Scalabilityistio.ioIstio 1.24 CPU and memory for a sidecar, a waypoint and a ztunnel at 1,000 requests per second, p99 latency per data-plane mode, and what drives istiod’s CPU.
7. Istio documentation: Sidecar or ambient?istio.ioThe two data-plane modes compared: resource use, L4 versus L7 features, Jobs, server-first protocols, key exposure and features not yet in ambient.
8. Istio’s ambient mode reaches General Availability in v1.24 (Istio blog, 2024)istio.ioThe GA announcement of 7 November 2024, the ztunnel and waypoint split, and the sidecar pain points (restarts, overhead) ambient mode removes.
9. Linkerd documentation: Architecturelinkerd.ioDestination, identity and injector components, the Rust micro-proxy, outbound versus inbound duties, iptables via linkerd-init, and native sidecars by default.
10. gRFC A27: xDS-Based Global Load Balancinggithub.comHow gRPC clients speak xDS directly with no proxy: the XdsClient, the bootstrap file and xds: target URIs; implemented in C-core, Java and Go.