Scalability and cost used to be separate conversations — one for the architecture review, one for finance — and at the architect level they no longer are. Cloud bills have become visible enough, and tunable enough, that an architect who can scale a system but cannot explain what that scaling actually costs, or who over-provisions "to be safe" without a plan to right-size later, is doing half the job. Interviewers use these questions to see whether a candidate treats capacity and cost as an ongoing engineering discipline rather than a one-time sizing exercise, and whether they understand the specific levers — autoscaling signals, caching, reservations, .NET's own runtime tuning — well enough to make a concrete recommendation instead of a general one. The ten questions below cover scaling strategies, autoscaling signals, capacity planning, caching and CDNs, FinOps practices, reservations and savings plans, and right-sizing .NET workloads specifically.

Q1 Walk through the difference between scaling up, scaling out and scaling to zero — when does each make sense for a .NET workload?#

Short answer: Scaling up (vertical scaling) gives one instance more CPU, memory or faster storage and is the simplest lever but has a hard ceiling and usually means downtime to apply; scaling out (horizontal scaling) adds more instances behind a load balancer and is the default strategy for stateless .NET services because it has no practical ceiling and can shed unhealthy instances without downtime; scaling to zero removes all instances when there is no traffic and restarts on demand, which suits bursty or infrequent workloads but reintroduces cold-start latency as a cost.

Vertical scaling still matters more than architects sometimes give it credit for: a poorly optimized service that needs ten small instances might need only three after a memory leak fix or a Server GC tuning pass, so "just add more instances" can mask a problem that a bigger or better-tuned single instance would have solved for less money. Horizontal scaling depends on the workload actually being stateless, or having state pushed out to a shared store (distributed cache, database, external session store) — a .NET service holding in-process session state or in-memory caches unique to that instance cannot scale out safely without sticky sessions, which themselves limit how evenly load balances and how gracefully an instance can be drained. Scale-to-zero is where the trade-off is sharpest: Azure Container Apps and Azure Functions on a consumption plan can both scale to zero, and KEDA-based scaling on AKS does the same for Kubernetes workloads, but every scale-from-zero event pays a cold start — JIT warm-up, dependency injection container build, first-request query plan caching — that Native AOT and pre-warming strategies can shrink but rarely eliminate entirely. The right choice is rarely "pick one": a production API is typically scaled out with a sensible minimum instance count for latency-sensitive traffic, while an adjacent batch or webhook-processing component on the same system scales to zero because its traffic is genuinely bursty and occasional added latency is acceptable.

What interviewers look for: a clear grasp of what each strategy actually solves and what it costs, plus the judgment to apply different strategies to different components of the same system rather than one blanket policy.

Common mistakes:

  • Defaulting to horizontal scaling for a workload that is expensive per-instance because of a fixable inefficiency, instead of investigating why it needs so many instances in the first place.
  • Choosing scale-to-zero for a latency-sensitive, frequently hit path without accounting for cold-start cost, then being surprised by tail-latency complaints.

Q2 What signals should drive autoscaling decisions, and why is CPU alone usually a bad primary signal for .NET services?#

Short answer: Autoscaling should react to the signal closest to the thing you actually care about — request latency, queue depth, or a custom business metric — because CPU utilization in a .NET service often lags or misrepresents real load: async I/O-bound work can saturate thread pool queues and blow up latency while CPU stays low, and a GC pause can spike CPU briefly without reflecting a sustained capacity problem either way.

Azure Monitor autoscale rules support both metric-based and schedule-based triggers, and the practical lesson is to combine them: schedule-based rules pre-scale ahead of known traffic patterns (a business-hours ramp, a known batch window) so the system isn't reacting to load it could have anticipated, while metric-based rules handle the unpredictable part. For the metric itself, request queue length or in-flight request count is usually a better primary signal for an ASP.NET Core service than CPU, because it directly reflects whether requests are backing up rather than a proxy for it. For message-driven workloads, queue depth or consumer lag is the natural signal, and on Kubernetes, KEDA turns exactly that kind of external metric — RabbitMQ queue length, Kafka consumer lag, an Azure Storage Queue depth — into a first-class autoscaling trigger by acting as a Kubernetes metrics server that feeds the Horizontal Pod Autoscaler, including scaling down to zero replicas when a queue is empty. Whatever signal you choose, autoscaling also needs sane cooldown periods and min/max bounds: too short a cooldown causes thrashing (scale out, then immediately back in, repeatedly), and too generous a max without a cost alert can turn a traffic spike into a bill spike with nobody watching.

What interviewers look for: specific reasoning about why CPU misleads for async, I/O-bound .NET workloads, and knowledge of real signal types (queue depth, request latency, consumer lag) and tools (Azure Monitor autoscale, KEDA) rather than a generic "scale based on load."

Common mistakes:

  • Using CPU as the only autoscaling signal for an I/O-bound service and being surprised when latency degrades while CPU utilization looks fine.
  • Setting an aggressive scale-out threshold with no matching cooldown, causing the fleet to thrash size up and down on normal traffic variance.

Q3 How do you do capacity planning for a system that has to handle both steady growth and unpredictable spikes?#

Short answer: Separate the two problems explicitly: steady growth is a forecasting problem solved with historical trend data and a planned, reviewed baseline capacity that you adjust on a cadence, while unpredictable spikes are an elasticity problem solved with headroom, autoscaling and load shedding, not by permanently provisioning for the worst case you can imagine.

Concretely, capacity planning starts from real utilization data, not intuition: look at p95 and p99 resource utilization and request rate over a meaningful window — weeks, not a single day — because a plan based on daily averages under-provisions for realistic peaks, and a plan based on the single worst moment you've ever seen over-provisions for every normal day in between. Build a baseline from that data with a deliberate safety margin (not from "double it to be safe," which is how quiet over-provisioning becomes permanent), and revisit it on a fixed cadence — quarterly is common — as a real review, not an afterthought when someone notices the bill. For the spike side, the architecture question is whether the system can degrade gracefully under unplanned load: circuit breakers and load shedding that reject or queue excess work predictably are cheaper and safer than trying to provision headroom for every conceivable spike, and they turn "the system fell over" into "the system briefly rejected some requests with a clear signal," which is a fundamentally different incident. Capacity planning for a known seasonal peak (a specific sale event, a fixed reporting deadline) deserves its own pre-planned runbook — pre-warm scaled-out capacity ahead of the exact known window rather than relying on reactive autoscaling to keep up with a spike that arrives faster than new instances can start and warm up.

What interviewers look for: a real forecasting process grounded in percentile data rather than guesswork, and a clear separation between planned growth (a capacity decision) and unplanned spikes (an elasticity and graceful-degradation decision).

Follow-up questions:

  • How would you plan capacity for a service whose traffic pattern you have no historical data for yet?
  • What would make you decide to pre-warm capacity ahead of a known event instead of trusting autoscaling?

Q4 How do caching and CDNs fit into a scalability strategy, and where do they introduce new failure modes?#

Short answer: Caching and CDNs move load away from the systems that are hardest and most expensive to scale — a database, an origin server — toward layers that are cheap to scale horizontally and close to the user, which is often the single highest-leverage scalability change available; the cost is a new failure mode class of its own: stale or inconsistent data, cache stampedes, and an origin that becomes fragile precisely because it has never had to handle real load once the cache is warm.

A CDN absorbs static assets and, increasingly, cacheable API responses at the network edge, cutting both latency and origin load for anything that doesn't need to be computed per-request. Inside the application, a distributed cache such as Redis (through .NET's HybridCache or IDistributedCache) protects a database from repeated identical reads, and an in-process cache protects against redundant work within a single instance for data that changes rarely. The failure modes that matter for a scalability conversation, specifically: a cache stampede, where a hot key expires and hundreds of concurrent requests all miss simultaneously and hammer the origin at once — mitigated with a short random jitter on expirations, or a "stale while revalidate" pattern where one request refreshes the value while others keep serving the slightly stale one; cache inconsistency across instances if invalidation isn't propagated correctly, which turns "the cache helped scalability" into "the cache served stale prices to half our users"; and origin fragility, where a system that has run for months behind an effective cache has quietly lost the capacity headroom to survive a cache outage, because nobody has load-tested the origin alone in a long time. A serious scalability review includes an explicit answer to "what happens if the cache disappears right now," not just "what happens when it's working."

What interviewers look for: framing caching as a scalability tool with its own operational risk, not a free win, and specific mitigations for stampedes and staleness rather than a vague "we cache things."

Common mistakes:

  • Treating cache hit rate as the only metric that matters, without tracking what happens to the origin during a cache miss storm.
  • Never testing the system with the cache disabled, so an outage in the cache layer becomes a full production incident instead of a graceful, if slower, degradation.

Q5 What is FinOps, and how do you operationalize cost accountability across engineering teams?#

Short answer: FinOps is the discipline of treating cloud cost as an engineering concern with the same visibility, ownership and iteration loop as performance or reliability — giving teams accurate, timely cost data broken down by what they actually own, making cost part of design and code review the way latency or security already are, and running a continuous cycle of optimizing and re-evaluating rather than a once-a-year budget exercise.

Operationalizing it starts with allocation: cost has to be attributable to a team, service or feature before anyone can be accountable for it, which in practice means consistent resource tagging enforced by policy (so cost reports aren't guesswork), and cost dashboards teams actually look at, not a finance-only report nobody on the engineering side sees. From there, the practical levers are the same ones an architect already reasons about for scalability — right-sizing, autoscaling instead of static over-provisioning, reservations and savings plans for predictable baseline load, caching to reduce expensive downstream calls — but FinOps adds the organizational half: a recurring review cadence where cost anomalies get the same triage attention as an incident, cost guardrails in CI/CD (a budget check that flags a pull request provisioning an unexpectedly large SKU), and, critically, cost as an explicit input to architecture decisions up front rather than a cleanup exercise after the bill arrives. The cultural shift that separates a mature FinOps practice from a cost-cutting exercise is that engineers see cost data close enough to real time, and close enough to their own service, that it becomes a normal part of engineering judgment — "this design will cost roughly X at our expected scale" said in a design review — instead of a surprise three weeks later in a spreadsheet someone else owns.

What interviewers look for: treating FinOps as a cross-functional, continuous practice with real mechanisms (tagging, allocation, guardrails, review cadence) rather than a synonym for "cut the cloud bill," and the insight that cost visibility has to reach engineers directly to change behavior.

Common mistakes:

  • Treating cost optimization as a one-time project instead of an ongoing loop with owners and a cadence.
  • Reporting aggregate cloud spend to leadership without ever breaking it down to a level individual teams can act on.

Q6 Reservations, savings plans and spot capacity — how do you decide what to commit to?#

Short answer: Match the commitment to how predictable the underlying usage is: reserve capacity (committing to a specific VM family and region for one or three years) for load you know will run steadily and won't change shape, use a savings plan (committing to a dollar amount of compute spend per hour, applied flexibly across eligible services, VM families and regions) for a baseline that is stable in total but shifts in composition, and use spot or low-priority capacity only for genuinely interruption-tolerant work, never for anything serving live production traffic without an explicit fallback plan.

The trade-off is flexibility versus discount depth: a reservation tied to one exact VM SKU in one region typically gives up some flexibility in exchange for the deepest, most predictable discount against that specific shape of usage, while a savings plan trades a little of that discount depth for the freedom to shift spend across different compute services and regions as the architecture evolves — a real consideration for a team still actively re-platforming, where locking into today's exact VM family for three years is a bet you may regret. Spot or low-priority VMs offer the steepest discounts of all in exchange for the provider being able to reclaim the capacity on short notice, which makes them a strong fit for batch processing, CI build agents, and stateless workers that checkpoint progress and handle an eviction notice gracefully, and a poor fit for anything stateful or latency-sensitive unless it's one replica among many behind a load balancer with on-demand capacity as a floor. A practical architect's approach: cover the durable, well-understood baseline with reservations or savings plans, leave genuine variability to on-demand and autoscaling, and use spot capacity only where interruption is a design input from day one, not an afterthought discovered during an actual reclamation event.

What interviewers look for: clear, correct differentiation between the three mechanisms and their respective flexibility/discount trade-off, plus explicit guardrails around when spot capacity is and isn't appropriate.

Common mistakes:

  • Reserving capacity for a workload whose shape is still actively changing, locking in the wrong SKU for years.
  • Running stateful production traffic on spot capacity with no fallback for reclamation, turning a routine capacity event into an incident.

Q7 How do you right-size a .NET workload that's over-provisioned, without risking a production incident?#

Short answer: Right-size from real utilization data over a representative window — p95/p99 CPU, memory and request latency, not a single snapshot or a raw average — reduce capacity in small, reversible steps with monitoring and an explicit rollback plan at each step, and validate the change under a real or realistic load test before trusting it in production unattended.

A common trap is sizing from average utilization, which looks comfortably low right up until the moment real peak traffic arrives and the now-smaller fleet can't absorb it; the right baseline is a percentile over weeks that includes your actual peak periods, with enough margin above that to absorb one bad day, not just a typical one. The safe sequence: pick the resource that's actually over-provisioned (CPU, memory, replica count — they are rarely all wrong by the same amount), reduce it by one step, and watch latency, error rate and saturation for a full business cycle before taking the next step, rather than jumping straight to the number the data suggests is theoretically sufficient. Right-sizing containerized .NET workloads needs one extra piece of care: since .NET Core 3.0, the garbage collector reads the container's memory limit — not the host's — to size its heap, defaulting to using a majority of that limit unless overridden with GCHeapHardLimitPercent, so shrinking a container's memory limit without understanding this can push the GC into much more frequent collections well before the process visibly runs out of memory, degrading latency before anything actually crashes. Automated recommendations (such as Azure Advisor's sizing suggestions) are a good starting signal, but treat them as a hypothesis to validate against your own percentile data and a load test, not an instruction to apply directly to production.

What interviewers look for: a safe, incremental, data-driven process rather than "look at average usage and shrink it," and specifically the container memory-limit and GC-heap-sizing interaction, which is a common source of self-inflicted incidents after a well-intentioned right-sizing pass.

Common mistakes:

  • Right-sizing from average utilization instead of a percentile that includes real peaks.
  • Shrinking container memory limits without accounting for how the .NET GC sizes its heap relative to that limit, causing GC pressure well before an out-of-memory crash would have made the problem obvious.

Q8 Your service handles ten times its normal traffic once a year during a predictable seasonal peak. How do you architect for it without paying for ten times the capacity year-round?#

Short answer: Treat the peak as a planned event, not a surprise: pre-scale ahead of the known window using scheduled autoscaling rules rather than relying purely on reactive metric-based scaling that can't add capacity as fast as the spike arrives, isolate the components that actually need to scale ten times from the ones that don't, and use elastic, consumption-priced capacity for the overflow instead of reserving year-round headroom for a few days of use.

Start by identifying which parts of the system genuinely need to scale for the peak — usually the request-handling tier and any queue consumers directly in the critical path — versus parts that don't, like an internal reporting service or an admin backend that sees no meaningful increase; scaling everything uniformly wastes money on components the peak never touches. For the tier that does need to scale, combine a schedule-based autoscale rule that pre-warms instances ahead of the known window (so the fleet is already sized correctly when traffic arrives, rather than racing to catch up) with a metric-based rule as a safety net for the case where the actual peak differs from the forecast. Favor consumption-priced or otherwise elastic capacity for the incremental scale-out specifically, so the extra cost is proportional to the days it's actually used rather than a reservation sized for a few days a year; a savings plan or reservation should cover only the steady-state baseline, never the peak multiplier. Finally, load-test at the actual target multiple before the real event, not just up to whatever traffic you've organically seen, since the first time a system experiences ten times normal load should never be the real event itself — a dedicated pre-event load test, plus a runbook covering what to watch and who's on call during the window, turns a known peak from a recurring fire drill into a routine operational event.

What interviewers look for: the pre-scaling and isolation-of-components instinct specifically, plus matching pricing model to usage pattern (elastic capacity for the spike, committed capacity only for the baseline) rather than a single blanket sizing decision.

Follow-up questions:

  • What would you do differently if the peak's exact timing were unpredictable within a known week rather than a known day?

Q9 How does .NET's own runtime behavior — garbage collection, the thread pool, JIT versus AOT — affect scalability and cost, and what would you tune first?#

Short answer: Server GC trades memory for throughput by giving each core its own heap and running collections in parallel, which is usually the right default for a scaled-out API but means memory footprint (and therefore cost per instance) scales with core count in a way Workstation GC doesn't; thread pool starvation from blocking calls inside async code quietly caps throughput well below what the hardware could otherwise sustain; and Native AOT shrinks both cold-start time and memory footprint, which matters disproportionately for scale-to-zero and high-density multi-tenant scenarios. The first thing to tune is almost always whichever of these is silently capping throughput per instance, because that directly multiplies the number of instances — and the cost — needed to hit a given target.

Server GC (ServerGarbageCollection in the project file, the default for ASP.NET Core apps) is right for most scaled-out services because it maximizes throughput, but it reserves a heap segment per logical core, so a container with a generous CPU limit and a small memory limit can end up GC-constrained in a way that isn't obvious from CPU metrics alone — this is exactly why the container memory-limit and heap-sizing interaction from right-sizing matters again here. Thread pool starvation is a quieter, frequently more expensive problem: a codebase with even a modest amount of blocking-on-async code (.Result, .Wait(), or a genuinely synchronous call inside a request handler) can exhaust available thread pool threads under load, causing request queuing and latency spikes that look identical to "we need more instances" in a dashboard but are actually fixed by finding and removing the blocking calls, not by scaling out. Native AOT compiles ahead of time with no JIT warm-up and a smaller memory footprint, which is the single highest-leverage change for a service that scales frequently from zero or runs at very high instance density, because it directly shrinks the cost of the thing that scale-to-zero otherwise taxes you for — cold start.

XML
<PropertyGroup>
  <ServerGarbageCollection>true</ServerGarbageCollection>
  <ConcurrentGarbageCollection>true</ConcurrentGarbageCollection>
</PropertyGroup>

What interviewers look for: a concrete diagnostic order — check for thread pool starvation and GC configuration before reaching for more instances — and the specific link between Native AOT and scale-to-zero cost, not a generic "the runtime affects performance."

Common mistakes:

  • Scaling out to compensate for thread pool starvation instead of finding and fixing the blocking calls causing it, which multiplies cost without fixing the root cause.
  • Leaving Server GC's default settings unexamined in a memory-constrained container instead of profiling actual heap behavior under realistic load.

Q10 How do you balance reliability — redundancy, multi-region deployment — against cost, and make that trade-off explicit to stakeholders?#

Short answer: Tie the reliability investment to an explicit, agreed target — a recovery time objective and recovery point objective per service tier, not a blanket "as available as possible" — and price each additional nine of availability concretely, because redundancy and multi-region deployment have a real, escalating cost curve, and the honest architectural answer is rarely "maximize reliability everywhere," it's "spend the redundancy budget where an outage is genuinely most expensive."

Not every component deserves the same reliability investment: a checkout path and an internal analytics dashboard do not have the same cost of downtime, and treating them identically either overspends on the dashboard or underspends on checkout. Multi-region active-active deployment roughly doubles compute cost at minimum, before accounting for cross-region data replication and the genuine engineering complexity of keeping state consistent across regions — a cost that is easy to justify for a payments path and hard to justify for an internal tool, which is exactly the kind of trade-off that should be made explicitly and revisited, not defaulted into uniformly because "more reliable is always better." The architect's job is to make the trade-off visible rather than deciding it unilaterally: translate "99.9% versus 99.99%" into what it actually costs in infrastructure and engineering effort, and what an hour of downtime actually costs the business for that specific service, so the decision is made by the people who own that business trade-off with real numbers in front of them, not buried in an infrastructure diagram nobody outside engineering reads. Revisit these tiers periodically too — a component that started as "internal and low-stakes" can become business-critical as the system evolves, and its reliability investment should be reviewed at the same cadence as its cost.

What interviewers look for: explicit RTO/RPO-driven tiering instead of one-size-fits-all reliability, and the communication skill to translate a technical trade-off into a business decision the right stakeholders actually make, which is a core part of the architect role beyond the technology itself.

Common mistakes:

  • Applying the same redundancy level to every component regardless of its actual cost of downtime.
  • Presenting reliability investment as a purely technical decision instead of a business trade-off with a concrete cost attached, leaving stakeholders unable to make an informed call.

Quick-Fire Round#

QuestionAnswer
Which scaling strategy has effectively no ceiling for stateless services?Horizontal scaling (scaling out).
Why is CPU often a poor primary autoscaling signal for async .NET services?I/O-bound load can saturate the system while CPU stays low.
What CNCF project scales Kubernetes workloads to zero based on external metrics like queue depth?KEDA.
What mitigates a cache stampede on a hot key's expiration?Jittered expirations or stale-while-revalidate.
Which commitment type ties a discount to one specific VM family and region?A reservation.
Which commitment type applies flexibly across compute services by dollar spend?A savings plan.
Since which .NET version does the GC size its heap from the container's memory limit?.NET Core 3.0.
What .NET runtime problem often masquerades as "we need more instances"?Thread pool starvation from blocking-on-async code.

How to Prepare#

  • Pull real utilization data (even from a side project) and compute p95/p99 over a multi-week window, so you can describe percentile-based capacity planning from practice, not theory.
  • Configure a schedule-based and a metric-based Azure Monitor autoscale rule on the same resource, and be ready to explain when each one fires.
  • Deliberately induce thread pool starvation with a blocking call under load, watch the effect on latency, and fix it — this is one of the most common "why don't we just scale out" traps in interviews.
  • Price out a reservation versus a savings plan for the same workload using current public pricing, and practice explaining the flexibility trade-off in your own words.
  • Prepare one real story where you reduced cost or footprint without a production incident, including exactly how you validated the change was safe before and after rolling it out.