System Design Interview: Distributed Rate Limiter in .NET

Q1 Design a rate limiter that enforces a single global limit — say 1,000 requests per minute per API key — across a fleet of 50 stateless API servers. Walk through your design.#

Short answer: Put the counters in a shared, fast store, almost always Redis, instead of process memory, because 50 independent in-process counters can't agree on a single global number. Each request does one atomic read-modify-write against that shared store, and the store's round-trip latency becomes the new cost of getting global accuracy.

Start from what breaks with the obvious wrong answer: give each of the 50 instances an in-process counter capped at 1,000 per minute, and the effective limit becomes up to 50,000 per minute, entirely dependent on how the load balancer happens to spread one API key's traffic. That is the built-in ASP.NET Core rate limiter's exact limitation — its counters are per process by design — and it is the reason a distributed rate limiter needs a store every instance can see and agree on. Redis is the default choice because it is fast, supports atomic operations natively, and most teams already run it for caching. The core design: each request computes a key from the API key and the current window or bucket state, issues one atomic command against Redis that both checks and updates the counter, and reads the result to decide allow or reject. "Atomic" is doing the load-bearing work here, since two servers issuing separate GET and SET calls can both read the same starting value and both allow a request that pushes the total over the limit. Latency budget matters too: every limited request now pays a network round trip to Redis before real work starts, so the store needs to sit close to the API tier, and the client needs a tight connection timeout so a slow Redis call never becomes a slow, hung request.

What interviewers look for: identifying the per-process-counter failure mode unprompted, naming atomicity as the actual requirement rather than just "use Redis," and volunteering the added latency as a trade-off rather than presenting a shared store as a free upgrade.

Common mistakes: describing "just use Redis" without explaining what makes the operation atomic; forgetting that every limited request now has a new, real dependency that can fail or add latency.

Follow-up questions:

• What would you do if the Redis round trip alone were eating too much of your latency budget?
• How would you avoid a single Redis instance becoming a new single point of failure for every limited request?

Q2 Compare token bucket, leaky bucket, fixed window and sliding window rate limiting. Which would you pick for a public API, and why?#

Short answer: Fixed window is simplest but allows up to double the limit in a short burst around a window boundary. Sliding window smooths that boundary burst at the cost of more state. Leaky bucket enforces a strictly constant outflow rate, which suits shaping traffic into a fixed-capacity downstream. Token bucket allows controlled bursts up to a cap while still bounding the long-run average, which is why it is the common default for a public API with legitimately bursty clients.

The detail people forget about leaky bucket is that it typically queues requests and releases them at a constant rate, trading burst tolerance for smoothing. That fits a scenario where you're protecting a downstream system that genuinely cannot absorb bursts, rather than a public API where a little burst tolerance is a feature, not a bug. For a public API, token bucket is the default because real clients aren't perfectly uniform — a client retries a batch of failed requests, or a mobile app syncs after being offline — and a bucket that allows a burst up to its capacity while still bounding the sustained average handles that gracefully. Fixed window either lets the same burst through unpredictably, right at a boundary, or blocks it arbitrarily mid-window.

What interviewers look for: the algorithm chosen to fit a stated scenario rather than recited as trivia; the leaky bucket versus token bucket distinction (constant output versus controlled bursts) is usually what separates a memorized answer from an understood one.

Common mistakes: treating leaky bucket and token bucket as the same idea with different names; picking fixed window "because it's simple" without naming its boundary-burst weakness as a real production risk.

Follow-up questions:

• How would a sliding window log differ from a sliding window counter in what it actually stores?
• Why might you choose fixed window anyway, despite its weakness?

Q3 Why do you need Lua scripting, or an equivalent atomic mechanism, to implement a rate limiter in Redis? What race condition does it prevent?#

Short answer: Because a rate limiter's check-then-act logic, read the current count, decide if it's under the limit, then write the new count, is not atomic across two separate Redis commands. Under concurrency, two requests can both read the same pre-increment value and both be allowed, silently letting the limit be exceeded. A Lua script runs as a single, uninterruptible unit on the Redis server, closing that gap.

Walk through the race concretely: two requests for the same key arrive within microseconds of each other on different API servers. Server A issues GET key, sees 999, under the 1,000 limit, and prepares to INCR and allow. Before A's INCR lands, Server B also issues GET key, also sees 999, and also prepares to allow. Both requests proceed, and the true count becomes 1,001. The limit was violated even though neither server did anything wrong individually; the read and the write just weren't atomic as a pair. A single Redis command is already atomic, so INCR alone, paired with EXPIRE NX to set a window's expiry once, is fine for a bare fixed-window counter. The moment your logic needs more than one command's worth of decision-making, such as token bucket math that reads the stored tokens, computes an elapsed-time refill, compares it to the requested amount, conditionally deducts, and writes back, you need every one of those steps to happen as one atomic unit. That is exactly what a Lua script, run through EVAL or EVALSHA, gives you: Redis executes the whole script on a single thread, uninterrupted by any other client's commands, so the read and the conditional write can never be split by another request's read.

What interviewers look for: the race condition explained as a concrete failure sequence rather than asserted, and recognition that a single Redis command is already atomic, with Lua specifically needed once logic spans more than one command.

Common mistakes: assuming "Redis is single-threaded, so it's all automatically safe," without recognizing that a client issuing two separate commands is not atomic against a single-threaded server, because another client's command can interleave between them.

Follow-up questions:

• What is EVALSHA, and why would you use it instead of EVAL on every call?
• How would you get the current time into a script without the script calling an external clock itself?

Q4 How does ASP.NET Core's built-in rate limiting middleware work, and where does it fall short for a truly distributed system?#

Short answer: Microsoft.AspNetCore.RateLimiting, built on System.Threading.RateLimiting, gives you four algorithms, fixed window, sliding window, token bucket and concurrency, as named or partitioned policies attached with RequireRateLimiting. It is a solid, well-tested building block, but its counters live in each process's memory, so across N instances behind a load balancer, a "1,000 per minute" policy actually allows up to N times that, and exactly how much over depends on how traffic happens to distribute.

builder.Services.AddRateLimiter(options =>
{
    options.RejectionStatusCode = StatusCodes.Status429TooManyRequests; // default is 503
    options.GlobalLimiter = PartitionedRateLimiter.Create<HttpContext, string>(http =>
        RateLimitPartition.GetTokenBucketLimiter(
            http.User.FindFirst("sub")?.Value ?? http.Connection.RemoteIpAddress?.ToString() ?? "anon",
            _ => new TokenBucketRateLimiterOptions
            {
                TokenLimit = 100,
                TokensPerPeriod = 20,
                ReplenishmentPeriod = TimeSpan.FromSeconds(10),
                QueueLimit = 0,
            }));
});

For a single instance, or as a coarse, best-effort safety valve even on a fleet, the built-in middleware is genuinely the right tool: it is free, well integrated with routing and Minimal APIs, and its per-instance concurrency limiter is a good local circuit breaker regardless of what a global limiter does elsewhere. For an exact, fleet-wide limit, it needs to be paired with, or replaced by, a shared-store implementation for the specific policies that must be precise. A common, pragmatic pattern is to run the built-in limiter's concurrency policy on every instance as a blunt local safety valve, while enforcing the precise, billed-tier limit through a Redis-backed limiter or at a gateway that sees all the traffic for a given key.

What interviewers look for: correctly locating the gap, per-process counters, rather than assuming the middleware is either "fine" or "useless." The nuanced answer is that it is the right default for local protection and the wrong tool alone for an exact global limit.

Follow-up questions:

• If an approximate limit is acceptable, how would you size a per-instance limit from a global target?
• What changes about this answer if the app autoscales aggressively during traffic spikes?

Q5 Design the response headers a rate-limited API should return. What does each communicate, and what goes wrong if they're inconsistent with reality?#

Short answer: Return the standard Retry-After header, in seconds, on every 429, and, to help well-behaved clients avoid ever hitting the limit, consider the limit, remaining and reset style headers many APIs converge on. Treat all of them as a courtesy to clients, not a contract you can violate silently, because a client that trusts a "remaining: 50" header and then gets rejected anyway will build broken backoff logic on top of it.

options.OnRejected = async (context, cancellationToken) =>
{
    if (context.Lease.TryGetMetadata(MetadataName.RetryAfter, out var retryAfter))
    {
        context.HttpContext.Response.Headers.RetryAfter =
            ((int)Math.Ceiling(retryAfter.TotalSeconds)).ToString();
    }

    await Results.Problem(statusCode: StatusCodes.Status429TooManyRequests, title: "Too many requests")
        .ExecuteAsync(context.HttpContext);
};

Fixed window, sliding window and token bucket leases in ASP.NET Core expose a RetryAfter value through lease metadata; the concurrency limiter does not, because it has no time basis, so a rejection there means "try again shortly," not "try again in exactly N seconds." The bigger design point is what happens when the advertised numbers lie: if a remaining-count header is computed from one instance's local counter while the actual enforcement is a shared Redis-backed limit, a client can see "50 remaining" and get rejected on its very next call, because the header and the enforcement point read from different sources of truth. Header values need to come from the same store that makes the allow-or-reject decision, or not be sent at all; a wrong remaining count is worse than no count.

What interviewers look for: Retry-After treated as the one header with a real behavioral contract, since clients use it to back off, and the header-versus-enforcement consistency issue named as a real design risk rather than an implementation detail.

Common mistakes: advertising limit and remaining headers computed from a different source than the one that actually enforces the limit; leaving the default 503 rejection status instead of 429, which trips load-balancer health assumptions.

Follow-up questions:

• What should a client do differently on a 429 with Retry-After versus a 503?
• How would you keep header values consistent when a request is rejected by a different layer, such as a gateway, than the one that would normally compute them?

Q6 How do you rate-limit fairly across tenants of very different sizes without starving small tenants or artificially capping large ones?#

Short answer: Give every tenant its own partition, its own counter or bucket keyed by tenant ID, instead of one shared bucket. Size each tenant's limit to its subscribed tier rather than an equal split, and where tenants genuinely share a pooled resource, use weighted fair queuing or a burst-into-shared-capacity scheme so a quiet tenant's unused budget can absorb a neighbor's spike without letting one tenant permanently dominate.

The naive failure mode is one shared limiter for "the API," which turns into a noisy-neighbor problem: a single large tenant with a traffic spike can consume the entire shared budget, starving every other tenant through no fault of their own. Partitioning by tenant, the same pattern ASP.NET Core's PartitionedRateLimiter uses, fixes the starvation but raises a new question: how do you size each partition? Equal shares waste capacity on small tenants and unfairly cap large paying customers; tier-based sizing, a token bucket capacity and refill rate scaled to the tenant's plan, is what most production systems actually do. For workloads that must share a fixed pool of downstream capacity rather than each tenant simply getting an independent slice, weighted fair queuing, where each tenant's requests are served proportionally to a configured weight rather than strictly first-come-first-served, keeps one tenant's burst from starving another even when the total pool is momentarily saturated, at the cost of real implementation complexity. A pragmatic middle ground many teams use: give each tenant a base guaranteed rate plus the ability to burst into an unused shared pool, so idle capacity isn't wasted but a spike from one tenant degrades gracefully into "back to your base rate" rather than "everyone is now rejected."

What interviewers look for: partitioning by tenant identified as the starting point, and, past that, a real answer to how you size the partitions rather than stopping at "give everyone their own bucket," since sizing is where the actual fairness design lives.

Common mistakes: partitioning by tenant but sizing every partition identically regardless of subscribed tier or usage pattern; conflating "fair" with "equal," which under-serves large legitimate customers and over-provisions idle ones.

Follow-up questions:

• How would you detect that a tenant's traffic pattern has permanently outgrown its tier, versus a temporary spike?
• What would you monitor to catch unfair partition sizing before a customer complains?

Q7 Your rate limiter's Redis cluster becomes unreachable. What should happen to traffic, and how do you decide?#

Short answer: Decide explicitly, in advance, whether the system fails open, letting traffic through unlimited while Redis is down, or fails closed, rejecting everything, because the default behavior of most client libraries, throwing an exception that bubbles up as an unrelated 500, is neither, and is worse than both. The right choice depends on what the limiter is actually protecting against.

var pipeline = new ResiliencePipelineBuilder()
    .AddTimeout(TimeSpan.FromMilliseconds(50))
    .AddCircuitBreaker(new CircuitBreakerStrategyOptions
    {
        FailureRatio = 0.5,
        SamplingDuration = TimeSpan.FromSeconds(10),
        BreakDuration = TimeSpan.FromSeconds(30),
    })
    .Build();

async Task<bool> IsAllowedAsync(string key, CancellationToken cancellationToken)
{
    try
    {
        return await pipeline.ExecuteAsync(
            async token => await CheckRedisLimiterAsync(key, token), cancellationToken);
    }
    catch (BrokenCircuitException)
    {
        return LocalFallbackLimiter.TryAcquire(key); // degraded and approximate, but not zero protection
    }
}

Wrap the Redis call in a circuit breaker so a failing dependency stops being called at all for a cool-down period instead of adding latency to every request while it recovers. Fail-open reasoning fits abuse-prevention or quota-style limiters, where letting some over-quota traffic through briefly is safer than taking the whole API down; fail-closed fits limiters protecting a fragile downstream, where unlimited traffic during the outage would cascade into a bigger outage than the rate limiter itself failing. A local fallback, an in-process concurrency limiter or a coarse in-memory fixed window that activates once the circuit trips, gives you "worse but not zero" protection instead of a binary open-or-closed choice. The Redis call's own timeout has to be tight and bounded regardless of which policy you pick, or a hanging connection turns a rate-limit check into the slowest part of every request.

What interviewers look for: fail-open versus fail-closed presented as a deliberate, context-dependent decision rather than a universal default, plus a concrete degraded-mode fallback instead of "the request just fails."

Common mistakes: letting an unhandled Redis exception become an unrelated 500 instead of a deliberate decision; setting no timeout on the Redis call, so a hanging dependency stalls every limited request.

Follow-up questions:

• How would you alert on "the limiter is in fallback mode" without that alert itself becoming noise during a brief Redis blip?
• Would your fail-open-versus-fail-closed answer change for a payments API versus a public read-only API?

Q8 Design a rate limiter for a multi-region deployment where clients can land in any region. How do you keep a global limit approximately correct without a single global point of failure?#

Short answer: Accept that a perfectly accurate global counter across regions is a trade-off you don't actually want, since it would require synchronous cross-region coordination on every request, trading away the availability and latency multi-region exists to buy. Instead, split the global budget across regions, statically or dynamically, and enforce each region's share locally, accepting some slack at the edges of the total.

Three mechanisms cover most real designs. A static split divides the global limit by expected traffic share per region, and each region runs its own local Redis-backed limiter against its slice; it's simple but wastes capacity if traffic is uneven or a region fails over. A dynamic, periodically synced split has each region report its local usage to a central store, or gossip with peers, every few seconds, and adjusts its local allowance accordingly, trading a bounded staleness window for better utilization. The third option is to deliberately accept approximate enforcement: do local-only enforcement per region with no synchronization at all, and accept that the true global total is, in the worst case, close to N times a single region's limit. That's often fine for a coarse abuse-protection limiter even though it wouldn't be acceptable for anything billing-accurate. The architecturally honest framing: distributed rate limiting across regions is the same choice as any distributed system spanning a partition-prone network. You can have a limit that is exactly correct, or one that stays available and low-latency during a network partition between regions, not both at the same instant.

What interviewers look for: the availability-versus-accuracy trade-off named explicitly, and a specific mechanism, static split, periodic sync, or accepted slack, rather than a hand-wave like "replicate Redis across regions."

Common mistakes: proposing synchronous cross-region Redis replication as if it solves this for free, without acknowledging the added latency, or reduced availability during a partition, that comes with it.

Follow-up questions:

• How would you rebalance region budgets after a regional failover moves traffic?
• What would make you choose strict-but-slower over approximate-but-fast for a specific system?

Q9 Walk through implementing a token bucket limiter directly against Redis. What does the atomic script need to compute, and how do you call it from a .NET service?#

Short answer: The script needs to, in one atomic step, read the bucket's stored token count and last-refill timestamp, compute how many tokens have accrued since then from the elapsed time and refill rate, cap the result at the bucket's capacity, check whether enough tokens exist for the request, and, only if so, deduct and persist the new state. Calling it from .NET is a single ScriptEvaluateAsync against StackExchange.Redis with the key and parameters passed as arguments.

private const string TokenBucketScript = """
    local key = KEYS[1]
    local capacity = tonumber(ARGV[1])
    local refillPerSec = tonumber(ARGV[2])
    local now = tonumber(ARGV[3])
    local requested = tonumber(ARGV[4])

    local data = redis.call('HMGET', key, 'tokens', 'ts')
    local tokens = tonumber(data[1]) or capacity
    local ts = tonumber(data[2]) or now

    local elapsed = math.max(0, now - ts)
    tokens = math.min(capacity, tokens + elapsed * refillPerSec)

    local allowed = 0
    if tokens >= requested then
        tokens = tokens - requested
        allowed = 1
    end

    redis.call('HMSET', key, 'tokens', tokens, 'ts', now)
    redis.call('EXPIRE', key, 3600)
    return { allowed, math.floor(tokens) }
    """;

async Task<(bool Allowed, int Remaining)> TryAcquireAsync(
    IDatabase db, string apiKey, int capacity, double refillPerSecond)
{
    double now = DateTimeOffset.UtcNow.ToUnixTimeMilliseconds() / 1000.0;
    var result = (RedisValue[])(await db.ScriptEvaluateAsync(
        TokenBucketScript, [$"rate:{apiKey}"], [capacity, refillPerSecond, now, 1]))!;

    return ((int)result[0] == 1, (int)result[1]);
}

Passing now as an argument from .NET, rather than reading Redis's own clock inside the script, keeps the script portable and testable, at the cost of trusting the caller's clock, which is fine when every caller is your own fleet behind reasonably synced NTP. EXPIRE on the key matters so idle buckets don't accumulate forever in Redis memory. In production, cache the script's SHA and call EVALSHA on subsequent invocations, falling back to a full EVAL if Redis reports NOSCRIPT because the script was evicted from its cache.

What interviewers look for: the full atomic sequence, read, compute elapsed refill, cap, check, deduct, persist, stated precisely rather than "run a Lua script," and awareness of the clock-source trade-off between caller-supplied time and Redis's own.

Common mistakes: computing the refill in application code and only using Redis for the raw counter, which reopens exactly the race condition the script exists to close; forgetting the expiry, leaking memory for keys that stop being used.

Follow-up questions:

• What happens to correctness if two API servers' clocks drift relative to each other?
• How would you unit test a script like this without a live Redis instance?

Q10 Where in a request's path, CDN, API gateway or individual service, should a rate limit actually live? Design a layered approach.#

Short answer: Put a coarse, cheap, IP- or connection-based limit at the edge to blunt obvious floods before they cost you anything, an identity-aware limit at the API gateway to enforce per-key or per-tenant business limits, and a resource-based concurrency limit inside each service to protect that service's own scarce resources regardless of what the outer layers already caught. Each layer answers a different question, and skipping one leaves a specific attack or overload scenario unprotected.

An edge layer sees raw connection and IP volume before authentication has even happened, so it's the only layer that can cheaply blunt a true flood, or a misbehaving client retry loop, before it costs any real compute; it typically cannot tell tenants apart, so its limits have to be coarse and generous. The gateway is where identity exists, since it's already terminating authentication, making it the natural home for a business-level limit like the "1,000 per minute per API key" from the first question: it's a single chokepoint that sees all of a key's traffic regardless of which backend instance eventually serves it, which is exactly the property that made global accuracy hard to get everywhere else. The service-level layer doesn't re-implement that same business limit; it protects that specific service's own capacity, a concurrency limiter bounding in-flight database calls, for instance, and it matters even when the gateway limit works perfectly, because one tenant operating well within quota can still send enough concurrent slow requests to exhaust a shared connection pool that every tenant depends on. Getting the layering backwards, only a coarse edge limit and nothing else, leaves per-tenant fairness and per-service resource protection both unaddressed; only a service-level limit and nothing at the edge means every flood request still pays the cost of reaching your infrastructure before being rejected.

What interviewers look for: each layer justified by what it uniquely protects against, not "defense in depth" as a slogan, and the specific insight that a service-level concurrency limit matters even when the gateway-level business limit is functioning correctly.

Common mistakes: implementing the same business-level limit redundantly at every layer instead of giving each layer a distinct job; relying solely on a gateway limit and leaving individual services with no local protection against a small number of tenants sending expensive, slow requests.

Follow-up questions:

• Which layer would you instrument most heavily for observability, and why?
• How should a request that's allowed at the gateway but rejected by a service-level concurrency limiter be surfaced to the client?