Service-to-Service Communication Interview Questions

Q1 How do you decide between synchronous and asynchronous communication for a given interaction between two services?#

Short answer: Ask whether the caller genuinely needs the result before it can proceed, and whether it can tolerate the callee being temporarily unavailable. If the caller needs an answer right now to continue — validating a payment before confirming an order to the user — you need synchronous communication; if the caller just needs a fact recorded and can proceed regardless of whether the callee happens to be up this second — notify billing that an order shipped — asynchronous messaging is both simpler and more resilient.

The core trade is availability coupling. A synchronous call makes the caller's availability a function of the callee's: if the callee is down, the caller's request fails too, immediately. An asynchronous message decouples that entirely — the caller can publish and move on even if the callee is completely down, because the message waits in the broker until the callee recovers. This is why "everything as synchronous REST calls" quietly turns a system of independently deployable services into one where an outage in any single service can cascade into every service that calls it, even indirectly through several hops.

The flip side is that asynchronous communication trades that availability for immediate consistency and simplicity — the caller genuinely doesn't know if or when the effect happened, which is fine for notify-style interactions and painful for anything the user is waiting to see the result of right now. A practical rule: use synchronous request/response for queries the caller is blocked on immediately, such as reading a product's price to render a page, and asynchronous messaging for commands and facts that don't require an immediate answer, such as "this order was placed, go do your part whenever you're ready."

What interviewers look for: the availability-coupling framing specifically, not just "sync is simple, async scales better," and a concrete rule for which interactions land on which side.

Follow-up questions:

• How would you make a slow synchronous dependency feel asynchronous to the end user without changing the underlying call?

Q2 Compare REST, gRPC and asynchronous messaging for service-to-service calls. What's your decision framework?#

Short answer: REST over HTTP/JSON is the default for synchronous calls that need broad interoperability, human-readable payloads and easy debugging with ordinary web tooling. gRPC is the choice for synchronous calls where performance and a strict, code-generated contract matter more than human readability — high-volume internal calls, tight latency budgets, or a genuine streaming requirement. Asynchronous messaging is the choice whenever the interaction doesn't need an immediate response at all, regardless of how fast either synchronous protocol is, because it removes the availability coupling REST and gRPC both share as synchronous protocols.

gRPC's real advantages are concrete, not just "it's faster." Protocol Buffers give you a strongly typed, contract-first schema with generated client and server code across languages; binary serialization that's meaningfully smaller and cheaper to process than JSON at high volume; and native support for client, server and bidirectional streaming that REST has no clean equivalent for. The cost is weaker natural interoperability with anything that isn't gRPC-aware — a browser calling directly needs grpc-web or a translating gateway — and payloads that aren't human-readable on the wire, which makes ad hoc debugging with curl or a browser's network tab harder than with plain JSON.

A workable framework in practice: default to REST/JSON for anything public-facing, anything a browser or third party calls directly, or anything where debuggability matters more than the last bit of latency; reach for gRPC for internal, high-volume or latency-sensitive service-to-service calls, especially where streaming is a real requirement rather than a hypothetical one; and reach for messaging whenever the interaction is actually a notification or a command that doesn't block the caller, regardless of which synchronous protocol you'd otherwise have reached for.

What interviewers look for: REST-versus-gRPC criteria that go beyond raw speed — contract strictness, streaming, interoperability, debuggability — and treating messaging as a decision on a different axis (sync versus async), not a third option competing on the same axis as REST and gRPC.

Q3 What is service discovery, and how does it work differently in Kubernetes versus a .NET-native approach like Microsoft.Extensions.ServiceDiscovery?#

Short answer: Service discovery answers "given a logical service name, what network address do I actually call right now," so services don't hardcode each other's IPs or hostnames, which would break the moment anything scales, restarts or moves. Kubernetes solves this at the platform layer — a Service object gets a stable DNS name that transparently load-balances across whichever pod replicas are currently healthy, entirely outside application code. Microsoft.Extensions.ServiceDiscovery solves the same problem at the application layer, resolving a logical service name to one or more endpoints through configuration and plugging directly into HttpClient, which matters when you're not on a platform that already provides DNS-based discovery, or you want the resolution logic explicit and testable in code.

The Kubernetes model is attractive because application code doesn't need to know discovery exists — you call a logical name and the cluster's DNS and networking layer handle the rest, which is powerful precisely because it's transparent. It also means the discovery mechanism sits entirely outside your application's own configuration and testing story, so running the same code outside Kubernetes removes that layer and you need a fallback.

The .NET-native model makes discovery an explicit, first-class part of the application's own configuration and HTTP pipeline, which is exactly what .NET Aspire's local development experience is built on — service references resolve through configuration Aspire injects during orchestration, and the same service-discovery wiring keeps working, backed by different resolvers, whether you're running locally, in a container without a mesh, or on a platform that also happens to provide its own DNS-based discovery. Knowing both models — and that they're not mutually exclusive, since application-level discovery can layer on top of a platform that does its own — is the senior-level answer.

What interviewers look for: platform-level discovery (transparent, outside the app) versus application-level discovery (explicit, inside the app's own pipeline) understood as two layers solving the same problem, not competing options where only one is correct.

Q4 What does a service mesh provide, and does adopting one mean you can delete your in-process resilience code?#

Short answer: A service mesh — sidecar proxies deployed alongside every service instance — provides uniform, network-layer implementations of cross-cutting concerns: mTLS between services, retries and timeouts at the proxy level, load balancing, and consistent telemetry for every call, without application code implementing any of it. It does not mean you can delete your in-process resilience code, because some of what belongs at the application layer — a business-aware fallback value, an idempotency key on a specific non-idempotent call, a circuit breaker threshold reflecting business-level knowledge of a dependency — isn't something a generic network proxy can decide for you.

What a mesh genuinely takes off your plate is the uniform, infrastructure-shaped part of resilience and security: every service gets mTLS and basic retry and timeout behavior at the proxy level without every team implementing it themselves, and the mesh's telemetry gives a consistent view of service-to-service traffic that would otherwise require every service to instrument itself the same way. That's real, valuable consolidation, and it's why meshes exist.

What a mesh can't replace is business-specific resilience logic — it doesn't know that a POST to a specific endpoint isn't idempotent and shouldn't be retried, or that a particular call's fallback should be a cached price rather than a generic error, or that a specific circuit breaker's threshold should be tighter because that dependency is known to be flaky under load. Most production setups that adopt a mesh keep application-level resilience for these business-aware decisions and let the mesh handle the uniform, protocol-level concerns underneath — the two layers are complementary, not a replacement of one by the other.

What interviewers look for: naming concretely what a mesh does handle and what it structurally can't, rather than treating "we have a mesh" as a reason application code needs no resilience thinking of its own.

Follow-up questions:

• What operational cost does adopting a mesh introduce that a team should budget for?

Q5 Explain mTLS in a service mesh. What is it actually protecting against, and what does it not protect against?#

Short answer: Mutual TLS means both sides of a connection present and verify a certificate, so a service doesn't just confirm it's talking to the right hostname, it cryptographically proves its own identity too — which lets a mesh enforce "only these specific services may call this service" at the network layer, and encrypts traffic so anyone observing the network path can't read or tamper with it in transit. It does not protect against a service that's legitimately allowed to call another one misusing that access — mTLS establishes and encrypts a trusted channel between two identities, it says nothing about whether the request those identities exchange is itself authorized to do what it's asking for.

The concrete gap is the difference between transport-level identity and application-level authorization. mTLS answers "is this really the orders service on the other end of this connection, and is the traffic private and untampered with" — it does not answer "should the orders service, authenticated as itself, be allowed to adjust stock levels for this specific warehouse on the inventory service." That remains a separate authorization decision, usually still needed at the application layer or through mesh-level authorization policies that go beyond plain mTLS. mTLS alone would happily let any correctly identified, mesh-enrolled service call any endpoint on any other mesh-enrolled service.

It's also worth being explicit that mTLS secures traffic inside the mesh; it says nothing about a compromised service's own code misusing data it's legitimately allowed to access, or about traffic that never goes through the mesh at all, such as a direct database connection or a call to an external third-party API. Treating mTLS as "our internal traffic is secure, full stop" is the mistake to avoid — it closes one specific, real gap, not every gap.

What interviewers look for: the transport-identity-versus-application-authorization distinction specifically, without overstating what mTLS alone secures.

Common mistakes: conflating "traffic is encrypted and mutually authenticated" with "every call is authorized to do what it's asking for."

Q6 How do you keep API contracts between services stable as both sides evolve independently?#

Short answer: Treat every contract — REST, gRPC or an event schema — as additive-only within a version, verified automatically rather than by convention: new optional fields and new endpoints ship freely, anything that removes or repurposes existing behavior gets a new version running alongside the old one, and consumer-driven contract tests catch a violation in the producer's own build before it reaches a shared environment.

The mechanics differ slightly per protocol, but the discipline doesn't. For REST/JSON, new fields are optional with sane defaults and existing endpoints never change meaning underneath a consumer. For gRPC, Protocol Buffers' numbered fields give you this almost by construction — adding a field with a new number is safe, reusing or renumbering an existing one is not — and generated clients make an accidental breaking change to a widely used field harder to ship unnoticed than with hand-written JSON models. For events, it's the same additive discipline messaging schema versioning requires, since an event becomes a contract the moment a second service consumes it.

What actually catches a violation before production is automated contract testing, not documentation. Each consumer publishes the subset of the contract it depends on, and the producer's pipeline runs those expectations against its real implementation on every change, so a breaking change fails a build in the producer's own CI instead of failing silently at runtime for a consumer that hasn't redeployed in months. This is the piece teams skip because it takes real setup, and it's exactly the piece interviewers probe for — a shared specification everyone reads but nothing enforces isn't a stable contract, it's a hope.

What interviewers look for: per-protocol mechanics — numbered protobuf fields, optional JSON fields, additive events — plus automated contract verification as what actually enforces stability, rather than a document.

Q7 What's the difference between a chatty and a chunky interface, and how do you tell which one you've built?#

Short answer: A chatty interface forces the caller to make many small round trips to accomplish one logical operation — fetch the order, then each line item separately, then each product's details separately — where every round trip pays the full network latency cost regardless of how little data it carries. A chunky interface returns everything the caller needs for a given use case in one call, shaped around what the caller actually does with the data rather than mirroring the callee's internal data model one entity at a time. You tell which one you've built by counting round trips per user-facing operation, not by inspecting any single endpoint in isolation.

Chattiness usually comes from designing an API as a literal mirror of a database schema — one endpoint per table — which is a natural single-process design but becomes expensive the moment each of those calls is a real network hop with its own latency, serialization and failure-mode cost. A page that needs an order, its line items and each product's current price and stock can turn into a dozen sequential calls if the API only exposes single-entity endpoints, and if any one of those round trips is slow, the whole page is slow, multiplied by however many round trips sit on the critical path.

The fix is designing endpoints around use cases rather than entities — an endpoint that returns an order with its line items and the specific product details a page actually needs, in one call, even if that endpoint's shape doesn't map one-to-one to a database table. This is also exactly the problem a backend-for-frontend or a gateway composition layer solves when chattiness is unavoidable at the service level: aggregate several chatty internal calls behind one chunky call the actual client makes, so the fan-out happens server-side on a fast internal network rather than over the client's much slower connection.

What interviewers look for: the round-trips-per-operation framing, and a specific fix — designing around use cases, or aggregating behind a BFF — rather than a vague "make fewer calls."

Common mistakes: designing an API as a literal one-endpoint-per-table mirror and being surprised when real usage needs a dozen sequential calls to render one screen.

Q8 How do you set and enforce a latency budget for a chain of service-to-service calls?#

Short answer: Start from the user-facing or business SLA — the total time this operation is allowed to take — and allocate it across the hops on the critical path based on each hop's realistic p99, not its average, leaving explicit headroom for retries and network variance. Then enforce it the way you enforce any budget: propagate a deadline through the chain, and continuously monitor actual per-hop latency against the allocation, not just check it at design time.

The average is the wrong number to budget against, because a chain's end-to-end latency is dominated by its slowest hop's tail, not the typical case. If a chain has five hops each with a p50 of 20ms but a p99 of 200ms, the chain's own p99 is much closer to "several hops having a bad moment simultaneously" than "five times 20ms" — and a retry anywhere in the chain can turn one slow hop into two attempts at that hop's own latency, compounding the effect further. Budgets need to be built from per-hop percentile data, not a single average multiplied by hop count.

Enforcement is where most budgets quietly die. A budget that exists only in a design document gets violated the first time a new hop is added or an existing one gets slower, unless something actually measures it continuously — per-hop latency dashboards with the budget drawn on them as a line, alerting when a hop's p99 creeps toward its allocation, and distributed tracing that lets you see, for a specific slow request, exactly which hop consumed the budget rather than guessing from aggregate metrics alone.

What interviewers look for: percentile-based, not average-based, budgeting, and enforcement treated as an ongoing, monitored process rather than a one-time design exercise.

Q9 When is gRPC a poor fit despite its performance advantages?#

Short answer: gRPC is a poor fit whenever the client isn't gRPC-native and can't easily become so — a browser calling directly without a translating gateway, a third-party integration partner who expects REST/JSON, or any public API where broad, low-friction interoperability matters more than the last bit of latency. It's also a poor fit when human-readable, easily debuggable payloads matter more than raw efficiency — an internal tool used for manual troubleshooting benefits far more from JSON you can inspect with any HTTP client than from a binary protocol that needs generated tooling just to read a request.

The browser case is the most common one in practice. gRPC's HTTP/2-based framing isn't something browsers speak the way they speak plain HTTP/JSON, so a browser client needs either grpc-web, which itself needs a proxy translating to real gRPC on the backend, or the API needs a REST/JSON facade in front of the gRPC services anyway — at which point you're maintaining two contracts for the same functionality unless the added complexity clearly pays for itself for that use case.

gRPC is also a worse fit than it first appears for genuinely public, long-lived APIs consumed by many unrelated external parties, where the friction of requiring generated clients and gRPC-aware tooling is a real adoption barrier compared to a URL and a JSON body any HTTP client already understands. The senior framing is that gRPC's strengths — contract strictness, binary efficiency, streaming — are specifically internal, service-to-service strengths, and they stop being advantages the moment the caller isn't an internal service under your own team's control.

What interviewers look for: the browser and public-API friction points named specifically, and gRPC's advantages recognized as scoped to internal, controlled-client scenarios rather than universal.

Common mistakes: picking gRPC for a public-facing API primarily because it's faster, without weighing the real adoption and tooling cost it imposes on external consumers.

Q10 How would you debug a service-to-service call that's slow, when each individual service reports healthy latency?#

Short answer: Trust distributed tracing over per-service metrics for this specific problem, because per-service dashboards each show their own processing time looking fine while the time actually lost is in the gaps between services — queueing before a request is picked up, connection establishment overhead, serialization, or a retry that each service's own metrics don't attribute to "this one slow request" at all. Pull the actual distributed trace for a slow request and look at where the time goes between spans, not just within them.

A trace that shows every service's own span as fast but a large gap between the end of one span and the start of the next is the tell — that gap is time spent outside any single service's own processing: network transit, a connection pool waiting for an available connection, a proxy queueing the request, or a client-side retry that re-executed the whole call and inflated total wall-clock time without any individual attempt looking slow in isolation. OpenTelemetry-instrumented services propagate trace context across calls specifically so this is visible as one connected trace instead of several disconnected sets of logs that each look fine on their own.

using var activity = ActivitySource.StartActivity("ProcessOrder", ActivityKind.Server);
activity?.SetTag("order.id", orderId);
// downstream HttpClient calls automatically propagate the trace context
// once OpenTelemetry's HttpClient instrumentation is registered

Once the gap is located, the usual suspects are connection pool exhaustion — too few connections configured for the real concurrency, so requests queue waiting for one to free up — DNS resolution repeated on every call instead of a pooled connection, or a client-side retry policy silently doubling or tripling the effective latency of a call that looks fast per individual attempt. None of these show up as "this service is unhealthy" on a per-service dashboard, because no single service actually did anything wrong in isolation.

What interviewers look for: looking at the gaps between spans specifically, not just span durations, and naming concrete causes — connection pooling, DNS, silent retries — that explain healthy services with a slow chain.

Common mistakes: checking each service's own latency metrics in isolation and concluding everything is fine without ever looking at a connected trace across the full call chain.