System Design Atlas

Backend & distributed systems field guide

Design for the load. Explain the trade-off.

A memory-first map of how requests move, systems scale, data stays correct, and failures stay contained.

Trace a request Use the interview loop → Return to DSA Atlas
  • 10visual maps
  • 4design phases
  • 8core trade-offs
design.md
# Requirements choose the architecture
load = estimate(qps, storage, bandwidth)
path = client → edge → service → data

bottleneck = measure(latency by hop + saturation)
if cacheable_repetition_limits_origin:
    test(cache)
if eligible_reads_saturate_data_node:
    test(read replicas)
if deferrable_bursts_create_backlog:
    test(durable queue + bounded workers)

state_tradeoffs()
design_for_failure()
measure_and_iterate()

01 · Request journey

Trace where one request can return, continue, or detach

Use this as a branching trace, not a mandatory serial stack. A cache hit returns early; deferred work leaves the synchronous response path only after a durable handoff.

  1. 01 · enterClientSet a deadline · retry only safe or idempotent operations
  2. 02 · locateDNS + edge routeResolve the endpoint · choose an eligible region or edge
  3. 03 · branchCDN lookupHit: return now · miss or dynamic request: continue to origin
  4. 04 · originLB + gatewayHealth-route · authenticate · authorize · limit
  5. 05 · computeServiceValidate the contract · execute domain logic
  6. 06 · branchCache lookupHit: return now · miss: read the authoritative store
  7. 07 · persistDatabaseRead on a miss · commit mutations under the required invariant
  8. 08 · optional branchDurable handoffQueue-only job or transactional outbox → acknowledge → worker
Memory hookAt each stage ask: can this request return, must it continue, or may work detach after durable acceptance?

02 · Scaling ladder

Scale only when the bottleneck asks

Each rung is a candidate intervention—not a required migration sequence. Measure the limiting resource, change the smallest useful boundary, and verify the result before climbing again.

Level 0

Single server

App + database. Simple, fast to change, one failure domain.

Start here
Level 1

Vertical scale

More CPU, memory, and disk. Easy but has a hard ceiling.

Machine is saturated
Level 2

Separate data

Move the database out. Scale compute and storage independently.

Resource contention
Level 3

Horizontal app

Stateless replicas behind a load balancer.

Compute QPS rises
Level 4

Cache + CDN

Reuse staleness-tolerant results or serve cacheable content nearer users.

Measured repetition or distance
Level 5

Replicate + shard

Copy eligible reads for capacity; partition ownership when storage, writes, or working set exceed one node.

Data-node capacity is the limit
Level 6

Async services

Durable queues smooth bounded bursts; workers isolate deferrable slow jobs. Shed or throttle sustained overload.

Deferrable work + drain headroom

03 · Architecture decision graph

Choose from the requirement, not the trend

Select the observed pressure to reveal what to diagnose, when a candidate pattern fits, and what it costs.

StartWhat is failing—or about to fail?Use evidence: latency, errors, saturation, or cost.
Diagnose, then choose

Break down tail latency by hop and inspect the query plan. If repeated cacheable reads dominate, test cache-aside and measure end-to-end benefit.

Fit conditions + trade-offs

04 · Data decision matrix

Match the store to the access pattern

The query shape, consistency need, and scale pattern matter more than labels.

Relations + transactions

Relational DB

Orders, payments, inventory, identity.

Joins · ACID · constraints
Flexible aggregate

Document DB

Profiles, catalogs, content records.

Nested data · schema evolution
Massive key access

Key-value

Sessions, counters, preferences.

Predictable lookup · easy partitioning
Relationship traversal

Graph DB

Social edges, fraud rings, knowledge graphs.

Multi-hop queries · connected data
High-write partitioned access

Wide-column

Telemetry, feeds, and time-bucketed records queried by partition key.

Partition-local clustering order · denormalized query model
Text relevance

Search index

Full-text search, autocomplete, filtering.

Inverted index · ranking
Copy vs splitReplication can increase read capacity and fault tolerance. Sharding increases total capacity.
Primary→
Replica AReplica B
Users A–HUsers I–PUsers Q–Z

05 · Distributed trade-offs

During a partition, choose the operation’s contract

When communication is lost or indefinitely delayed, an operation cannot guarantee both atomic consistency and an eventual valid response to every request received by a non-failing node. Make that failure behavior explicit instead of labeling the whole product “CP” or “AP.”

CAtomic consistencyone real-time-respecting order
ACAP availabilityevery request to a non-failing node eventually gets a valid response; no finite latency bound
PPartition tolerancenetwork can split
When
partitioned
Prefer consistencyPayments · inventory · locks

Reject or delay operations rather than expose conflicting truth.

Prefer availabilityFeeds · likes · presence

Serve possibly stale data and reconcile later.

Practical noteChoose behavior per operation

When communication is lost or indefinitely delayed, an operation cannot guarantee both atomic consistency and an eventual valid response at every non-failing node. Reject or wait to preserve atomic semantics, or respond from reachable state and reconcile under a weaker model; practical systems may sacrifice some of both.

Formal basis: Gilbert and Lynch’s CAP review ↗. Consistency models and session guarantees are a toolbox—not one total ordering of “strength.”

06 · Fast paths and slow paths

Cache repeated work. Queue deferred work.

Cache-aside read

Fast path

  1. 1Read cache
  2. Hit?
  3. 2AReturn value
  4. 2BRead database
  5. 3Populate cache + TTL

Remember: invalidation, stampede protection, eviction, and stale-data tolerance.

Asynchronous work

Slow path

  1. 1Validate + operation ID
  2. 2Durably enqueue, or commit state + outbox
  3. 3Acknowledge after durable acceptance
  4. 4Relay + worker consume
  5. 5Bounded retry → DLQ

Remember: a job-only request may acknowledge after the broker confirms durable acceptance. If a database mutation must emit an event, commit business state and an outbox record in one local transaction, then relay it. At-least-once delivery means consumers must tolerate duplicates—typically with idempotent effects, deduplication state, or an atomic inbox/write transaction.

07 · Reliability stack

Assume every dependency will fail

Layer defenses so one slow or broken service does not become a system-wide outage.

ObserveMetrics · logs · tracesKnow what is failing and who is affected.
BoundTimeouts · deadlinesNever wait forever.
RetryBackoff · jitter · budgetRetry transient faults without creating a storm.
IsolateCircuit breaker · bulkheadStop cascading failure.
DegradeFallback · stale read · shed loadPreserve the most important user path.
RecoverReplication · backup · failoverRestore service and data.
Reliability contractSLImeasured behavior→SLOtarget over a defined window→Error budget1 − SLO target over that window

08 · Interview loop

Four phases. One coherent story.

Alex Xu’s framework is effective because it keeps the conversation requirement-led and trade-off-aware.

  1. 01 · UnderstandClarify scope

    Users, use cases, read/write ratio, latency, consistency, geography.

    Output: functional + non-functional requirements
  2. 02 · EstimateSize the system

    QPS, peak factor, object size, storage growth, bandwidth, cache size.

    Output: order-of-magnitude numbers
  3. 03 · DesignDraw the main path

    APIs, data model, components, request flow, then the critical deep dive.

    Output: high-level architecture
  4. 04 · DefendFind bottlenecks

    Failure modes, scaling plan, consistency, observability, security, cost.

    Output: explicit trade-offs
Back-of-envelope

Use powers of ten, not false precision

Average QPS
daily requests ÷ 86,400
Peak QPS
average × peak factor
Write ingress
mutations/sec × bytes/mutation
Live logical data
active records × average bytes/record, or net retained creations/sec × bytes/record × retention
Append-log storage
events/sec × bytes/event × retention
Response egress
response QPS × response bytes

Do not treat every update as a newly retained record. Then add indexes, versions, replication, metadata, allocator overhead, compression, backups, protocol overhead, and request ingress as separate assumptions.

09 · System design thinking flow

Move from requirements to justified architecture

Do not begin with components. Each step produces evidence for the next decision.

StartWhat user problem must the system solve?Name the core use cases and what “good” means.
  1. 01
    Clarify scope

    Actors, actions, data, exclusions, and priority journeys.

    Output: functional requirements
  2. 02
    Set quality targets

    Latency, availability, consistency, durability, freshness, security, and cost.

    Output: non-functional requirements
  3. 03
    Estimate scale

    Users, QPS, peak factor, read/write ratio, object size, retention, and geography.

    Output: order-of-magnitude load
  4. 04
    Define contracts

    APIs, events, identifiers, pagination, idempotency, and error behavior.

    Output: system boundary
  5. 05
    Model the data

    Entities, access patterns, indexes, ownership, lifecycle, and consistency boundary.

    Output: source of truth
  6. 06
    Draw the simplest flow

    Trace one write and one read before adding scale mechanisms.

    Output: high-level design
  7. 07
    Find the bottleneck

    Check compute, memory, network, storage, contention, hot keys, and fan-out.

    Output: evidence for scaling
  8. 08
    Add one fitting pattern

    Cache, replicate, shard, queue, index, stream, batch, or move to the edge.

    Output: targeted improvement
  9. 09
    Design failure behavior

    Timeouts, retries, idempotency, failover, degradation, recovery, and observability.

    Output: resilient architecture
  10. 10
    Recheck trade-offs

    Does the final design still satisfy correctness, scale, operations, and cost?

    Output: defensible decision
Before choosingWhat factor dominates?
TrafficHow much and how bursty?

Average and peak QPS, concurrency, seasonality, and growth.

DataHow large and how long?

Record size, retention, growth, relationships, and access patterns.

Access shapeWhat makes each path expensive?

A ratio chooses nothing alone: check repetition, staleness tolerance, query cost, hot keys, contention, ordering, and durability.

LatencyWhere is the deadline?

Interactive requests, background jobs, tail latency, and geographic distance.

CorrectnessHow stale may data be?

Consistency boundary, ordering, uniqueness, transactions, and conflict policy.

AvailabilityWhat may stop working?

SLO, redundancy, graceful degradation, RTO, and RPO.

TopologyWhere are users and data?

Regions, residency, edge delivery, replication lag, and network partitions.

SecurityWho may do what?

Authentication, authorization, encryption, abuse, privacy, and auditability.

OperationsCan the team run it?

Observability, deployment, migration, on-call load, and failure isolation.

CostWhat is worth optimizing?

Compute, storage, bandwidth, managed services, and engineering complexity.

Bottleneck catalog

Find the constrained resource before choosing a pattern

A bottleneck is the resource or coordination point that limits throughput, raises latency, or increases failures as load grows.

ComputeCPU saturation
Symptoms
High CPU, rising request latency, run queue growth, timeouts.
Causes
Expensive serialization, compression, ranking, encryption, loops, or too few instances.
Measure
CPU utilization, CPU/request, profiling, load average, p95/p99 latency.
Typical moves
Optimize hot code, cache results, batch work, add stateless replicas, move heavy jobs async.
MemoryRAM pressure
Symptoms
Garbage-collection pauses, swapping, OOM kills, cache eviction, unstable latency.
Causes
Leaks, oversized caches, large object graphs, unbounded queues, or poor batching.
Measure
Working set, heap, allocation rate, GC pause, page faults, eviction rate.
Typical moves
Bound memory, stream data, fix leaks, reduce object size, tune cache policy, scale out.
NetworkBandwidth or connection limit
Symptoms
Slow transfers, packet loss, retransmits, connection failures, cross-region delay.
Causes
Large payloads, chatty APIs, media delivery, connection churn, or distant dependencies.
Measure
Bytes/sec, packets/sec, open connections, RTT, retransmits, payload size.
Typical moves
Compress, paginate, batch, use CDN, pool connections, colocate services, reduce round trips.
Storage I/ODisk throughput or IOPS
Symptoms
Slow queries, high I/O wait, queue depth growth, compaction or checkpoint stalls.
Causes
Random reads, write amplification, scans, insufficient indexes, or small synchronous writes.
Measure
IOPS, throughput, I/O wait, disk latency, queue depth, cache-hit ratio.
Typical moves
Add indexes, batch writes, cache reads, partition data, use SSDs, tune compaction.
DatabaseQuery or connection saturation
Symptoms
Lock waits, slow queries, exhausted pool, replica lag, deadlocks.
Causes
N+1 queries, missing indexes, hot rows, long transactions, joins at scale, too many clients.
Measure
Query latency, rows scanned, locks, active connections, buffer hit rate, replication lag.
Typical moves
Fix queries/indexes first; then cache, pool, replicate reads, partition writes, or precompute.
ContentionLocks and shared state
Symptoms
Throughput stops scaling with replicas, lock waits rise, retries and conflicts grow.
Causes
Global counters, hot records, coarse locks, centralized coordinators, serial sections.
Measure
Lock duration, conflict rate, retry count, queue time, time inside critical sections.
Typical moves
Partition ownership, reduce lock scope, use optimistic concurrency, local aggregation, append logs.
Hot keySkewed traffic
Symptoms
One cache node, shard, tenant, or celebrity account overloads while others remain idle.
Causes
Uneven partition key, viral content, time-based keys, or a small set of popular records.
Measure
Per-key QPS, per-partition load, top tenants, key frequency distribution.
Typical moves
Replicate hot data, request coalescing, key salting, adaptive partitioning, per-tenant limits.
QueueConsumer lag
Symptoms
Growing backlog, stale results, delayed notifications, increasing end-to-end time.
Causes
Slow consumers, poison messages, downstream throttling, too few partitions or workers.
Measure
Queue depth, oldest-message age, consumer lag, processing time, retry and DLQ rate.
Typical moves
Scale consumers, add partitions, batch, apply backpressure, isolate poison messages, shed work.
DependencySlow downstream service
Symptoms
Thread or connection pools fill, tail latency spreads, cascading failures begin.
Causes
Unbounded waits, retry storms, overloaded shared dependency, or synchronized traffic.
Measure
Dependency latency/error rate, in-flight requests, timeout rate, pool utilization.
Typical moves
Deadlines, bounded retry with jitter, circuit breaker, bulkhead, fallback, load shedding.
CoordinationSingle leader or metadata service
Symptoms
Leader overload, failover pauses, serialized writes, control-plane instability.
Causes
All writes or leases pass through one node; excessive consensus or metadata chatter.
Measure
Leader CPU/QPS, commit latency, election count, quorum latency, metadata request rate.
Typical moves
Partition leaders, lease/range allocation, reduce coordination, cache metadata, batch commits.
External limitsQuota or rate ceiling
Symptoms
429 responses, throttling, rejected sends, sudden latency from provider backoff.
Causes
Third-party API quotas, cloud service limits, account-level caps, or abuse protection.
Measure
Quota consumption, 429 rate, retry-after time, provider latency and errors.
Typical moves
Client-side limiting, caching, batching, queues, quota partitioning, graceful degradation.
CostEconomic bottleneck
Symptoms
System performs correctly but cost grows faster than users or revenue.
Causes
Low utilization, over-replication, excessive egress, inefficient queries, retained cold data.
Measure
Cost/request, cost/user, egress bytes, utilization, storage tier, idle resources.
Typical moves
Right-size, tier storage, reduce egress, improve cache hit rate, batch, autoscale, simplify.
Diagnose in order
  1. ObserveWhich SLI is degrading?
  2. LocalizeWhich stage adds queue time?
  3. ConfirmWhich resource saturates with load?
  4. Change one thingApply the smallest fitting pattern.
  5. VerifyDid latency, throughput, errors, and cost improve?

Pattern selector

Follow the workload signal to a candidate design

These are starting hypotheses. Confirm them against consistency, failure, operational, and cost requirements.

Repeated hot readsCache-asideCheck TTL, invalidation, stampede, and stale-data tolerance.
Mostly static global contentCDN / edge cacheCheck purge speed, personalization, residency, and origin protection.
Compute is saturatedStateless replicas + load balancerExternalize sessions and verify downstream capacity.
Read database is saturatedRead replicasAccept or control replication lag and read-after-write behavior.
Data or writes exceed one nodeShardingChoose a stable key; avoid hot partitions and cross-shard work.
Deferrable burst exceeds synchronous capacityQueue + workersProve durable acceptance, idempotency, bounded backlog, drain time, backpressure, and DLQ.
Many consumers need changesEvent stream / pub-subDefine schema evolution, replay, delivery semantics, and ownership.
Full-text or relevance queriesSearch indexAccept asynchronous indexing and reconcile with the source of truth.
Heavy relationship traversalGraph modelUse only when multi-hop access dominates simpler key or relational queries.
Repeated aggregate over historyPrecompute / materialized viewTrade freshness and write work for predictable read latency.
Huge work not needed immediatelyBatch pipelineDefine completion SLA, checkpointing, partitioning, and reruns.
Same event may arrive twiceIdempotent consumerUse stable operation IDs, deduplication state, or naturally idempotent writes.

Common pattern handbook

How the patterns work in a real design

For every pattern, state the problem it solves, where it sits in the flow, and the new failure mode it introduces.

01Load balancingSpread requests across healthy compute replicas

Use when: one application instance cannot handle peak traffic or must not be a single point of failure.

How: keep services stateless, register healthy instances, choose a routing policy, and drain instances during deployment.

Watch: uneven long-lived connections, sticky sessions, slow health checks, and overloaded downstream services.

02Cache-asideServe repeated reads without repeating database work

Use when: reads repeat, values tolerate bounded staleness, and the database is the source of truth.

How: read cache → on miss read database → populate with TTL. On write, update the database and invalidate or refresh the key.

Watch: stale values, cache stampedes, hot keys, eviction, and low hit rates that add an extra network hop.

03CDN / edge deliveryMove content and request handling closer to users

Use when: static assets, media, downloads, or cacheable responses serve geographically distributed users.

How: define cache keys and headers, choose TTLs, protect the origin, and provide an invalidation strategy.

Watch: personalized content leakage, slow purge, origin stampede, and data-residency constraints.

04Read replicationCopy data to increase read capacity and tolerate node failure

Use when: measured read load exhausts a data node after query and index fixes, eligible reads can accept the chosen replica semantics, and the primary retains write and replication headroom.

How: route writes to the primary and eligible reads to replicas; send read-after-write traffic to the primary or track a consistency token. Availability improves only when routing, health detection, quorum or promotion rules, and tested failover can use the copies correctly.

Watch: replica lag, stale reads, failover correctness, synchronous-replication write availability, and multiplying expensive queries across replicas.

05Sharding / partitioningSplit ownership so data and writes scale beyond one node

Use when: storage, write throughput, or working set exceeds a single database node.

How: choose a high-cardinality stable partition key, route requests consistently, and design rebalancing before growth demands it.

Watch: hot partitions, cross-shard joins/transactions, resharding, global indexes, and unique ID generation.

06Queue + worker poolBuffer bounded bursts and remove deferrable work from request latency

Use when: work may safely complete later and bounded bursts, retries, or a slower dependency need durable buffering. Long-run worker capacity must exceed admitted arrival rate, or admission control must cap it; a queue does not fix sustained overload.

How: assign a stable operation ID and validate synchronously. For a job-only API, wait for durable broker acceptance before acknowledging; when a database mutation must emit work, commit business state plus an outbox record atomically and relay it. Process with horizontally scaled, duplicate-tolerant consumers; bound transient retries and isolate poison jobs in a DLQ.

Watch: database/broker dual-write gaps, ambiguous publish outcomes, duplicate delivery, queue lag, unbounded retries, ordering requirements, and backpressure.

07Event stream / pub-subDistribute state changes to independent consumers

Use when: multiple systems react to the same event, consumers evolve independently, or replay is valuable.

How: publish immutable domain events with versioned schemas; partition by the ordering key; consumers keep checkpoints and idempotent state.

Watch: unclear ownership, schema breaks, reordering across partitions, replay load, and eventual consistency.

08Database indexTrade write/storage cost for targeted query speed

Use when: a frequent selective query otherwise scans too many rows.

How: design for one high-value query and the chosen database/index type. For a typical B-tree, consider leading equality predicates, the first range predicate, required sort order, and whether included columns enable a covering read; verify with the actual query plan and keep the index narrow.

Watch: engine-specific rules, slower writes, extra storage, unused indexes, low-selectivity fields, and indexes whose leading columns or ordering do not match the query.

09Materialized view / precomputationMove repeated aggregation from read time to write or background time

Use when: dashboards, feeds, counts, or rankings repeatedly derive the same expensive result.

How: update the derived view from transactions or events, store a rebuild path, and expose freshness explicitly.

Watch: stale or divergent views, write amplification, difficult backfills, and double-counted events.

10Rate limitingProtect capacity and enforce fair usage

Use when: clients can overload an API, abuse is possible, or downstream quota must be protected.

How: choose the identity and scope, apply token bucket or sliding-window logic near the edge, and return clear retry information.

Watch: distributed counters, clock/window boundaries, shared-NAT users, fail-open versus fail-closed behavior.

11Circuit breaker + bulkheadStop a failing dependency from consuming every resource

Use when: a remote dependency can become slow or unavailable and requests otherwise wait or retry indefinitely.

How: enforce deadlines, isolate pools per dependency, open the circuit after a threshold, probe recovery, and provide fallback behavior.

Watch: thresholds that flap, hidden failures, stale fallback data, and breakers without upstream load shedding.

12Idempotency + deduplicationMake retries safe when outcomes are uncertain

Use when: clients, queues, or networks may retry create, payment, or mutation requests.

How: attach a stable operation key, atomically store its result with the mutation, and return the original result on duplicate requests.

Watch: key scope, retention period, payload mismatch, races, and operations whose side effects happen outside the transaction.

13Saga / compensating transactionCoordinate a multi-service workflow without one distributed transaction

Use when: a business operation spans independently owned services and partial completion must be repaired.

How: persist each step, trigger the next step, define compensating actions, make every handler idempotent, and expose workflow state.

Watch: compensation that cannot truly undo effects, long-running state, concurrent workflows, and poor observability.

14Consistent hashingDistribute keys while minimizing movement when nodes change

Use when: partitioned caches or stores frequently add/remove nodes and simple modulo hashing would remap most keys.

How: map keys and nodes into an ordered hash space so membership changes remap only part of the keyspace; use virtual nodes or another balancing method. Replication to successor nodes is an optional layer, not a property of consistent hashing itself.

Watch: skew, coordinated membership, replication during churn, and the fact that hot keys remain hot.

15Leader election / leaseEnsure one active coordinator for a scoped responsibility

Use when: one node must schedule work, own a partition, allocate IDs, or perform a singleton task.

How: acquire a consensus-backed lease and attach a monotonically increasing fencing token to every protected operation. The protected resource must reject stale tokens; do not rely only on a paused former owner noticing lease loss.

Watch: split brain, clock assumptions, long pauses, stale leaders, and making the leader a throughput bottleneck.

Gate 1Can the simple design meet the estimates?

Yes: stop adding components. No: name the measured or projected bottleneck.

Gate 2Does the pattern preserve correctness?

State what becomes stale, duplicated, reordered, unavailable, or harder to recover.

Gate 3Is the complexity operationally affordable?

Every new datastore, queue, region, and cache adds failure modes and ownership.

Gate 4How will you prove it works?

Choose SLIs, load tests, alerts, capacity thresholds, and rollback signals.

Design sentenceBecause requirement R and estimate E create bottleneck B, choose pattern P, accepting trade-off T, measured by M.

10 · Quantitative design workbench

Turn estimates into architecture pressure

Interview arithmetic is not a capacity promise. It is a fast way to expose the dominant path, choose a first design, and name what must be measured before production.

Worked example · flash-sale checkout

Follow one assumption until it changes the design

Assume two million purchase attempts arrive over ten minutes, peak traffic is five times the average, the synchronous path targets 250 ms, responses average 2 kB, and one percent of attempts become orders.

  1. Expose the peak2,000,000 ÷ 600 s ≈ 3.3k attempts/s average; a stated 5× burst means approximately 16.7k attempts/s at peak.
  2. Estimate in-flight workIf 16.7k/s is sustained and the mean time in the same stable system boundary is 0.25 s, Little’s Law gives about 4,200 requests in flight on average. Queueing inside that boundary is already part of the mean time; if 250 ms is only a p99 target, treat this multiplication as a rough capacity envelope rather than a theorem-backed average.
  3. Separate browse from commitAt a 1% conversion assumption, the durable order path is about 167 orders/s. Inventory reservation, payment intent, and idempotency need stronger semantics than catalog reads.
  4. Price the network path16.7k responses/s × 2 kB is about 33 MB/s (≈267 Mb/s) of response payload before request bytes, TLS, headers, replication, or observability traffic.
  5. Stop retry multiplicationWith one original attempt plus two retries at each of three nested layers, a single request can cause 3³ = 27 downstream attempts. Put a bounded retry policy at one owning layer and add backoff, jitter, and a retry budget.
Make correctness local: write the invariant, choose its smallest consistency scope, then test the ambiguous outcome.
InvariantScopeCandidate mechanismFailure test
One local payment intent/result per scoped idempotency keyMerchant + operation + idempotency keyAtomically persist local intent, result, and outbox; propagate the same authenticated key to a provider that supports idempotency. If the remote outcome is unknown, query and reconcile before retry.The provider charge succeeds but the response is lost; prove lookup/reconciliation prevents a second charge and rejects a conflicting payload for the same scoped key.
Sellable inventory never drops below zeroSKU or reservationTransactional conditional decrement, or a durable expiring reservation with an explicit release path.Many buyers race for the last unit; a checkout crashes after reservation but before payment.
Messages stay ordered inside one conversationConversationPer-conversation sequence numbers or one ordered partition; reject, buffer, or reconcile gaps.Two devices send concurrently while one reconnects and replays an earlier request.
A short alias names at most one URLAlias keyConditional insert or unique constraint at the authoritative store.Two creators claim the same custom alias at the same time.
A feed meets its freshness objectiveFeed readEvent watermark, consumer-age SLI, and a declared stale-read policy.Pause a consumer, build backlog, recover, and prove drain time without overloading the source.
  1. Accept mutationValidate deadline, identity, idempotency key, and expected version.
  2. Commit onceWrite business state, saved result, and outbox record in one local transaction.
  3. AcknowledgeA retry reads the stored result even if the first response disappeared.
  4. Relay eventA relay may publish more than once; the consumer must tolerate duplicate delivery.
  5. Build derived stateTrack watermark, replay progress, poison work, and reconciliation evidence.
Design the failure contract before failover. A useful row names user behavior, overload guardrail, data risk, detection, and the proof of recovery.
FailureUser-visible behaviorGuardrail + data riskEvidence
Cache unavailableServe a bounded stale result, shed optional reads, or bypass only while origin headroom remains.Prevent a cache miss storm from saturating the source of truth; no silent indefinite staleness.Cache-disabled peak test; hit rate, origin QPS, saturation, and stale-age SLI.
Primary store unavailableReject or delay consistency-sensitive writes, or enter a tested promotion path.Fence stale leaders; state RPO and which reads may become stale during promotion.Leader-loss exercise; commit latency, replica lag, stale-token rejection, recovery time.
Queue backlogPreserve accepted durable work while exposing freshness degradation.Slow noncritical producers, bound retries, and isolate poison work from the main queue.Oldest-message age, lag, DLQ rate, drain-rate estimate, and a peak-backlog recovery test.
Region unavailableRoute only to healthy, pre-provisioned capacity; declare reduced features if necessary.State RTO, RPO, consistency impact, DNS/routing delay, and dependency locality.Regional evacuation exercise with measured capacity, data convergence, and rollback.
Bad deploymentHalt the canary and restore the last compatible code, schema, and configuration.Limit blast radius and avoid a rollback that cannot read forward-written data.SLO burn correlated with change events; rollback-time and compatibility rehearsal.

Interview close: leave a decision ledger, contract sheet, capacity sheet, failure matrix, observability plan, and open-risk register. “Exactly once” is useful only after naming the source, processing boundary, destination, and replay behavior.

Correctness

Name the invariant

Examples: one local payment intent per scoped key; one external charge only with provider idempotency plus uncertain-outcome reconciliation; no negative sellable inventory; ordered messages within one conversation. Then choose the smallest transaction or consensus boundary that protects each claim.

Overload

Choose who waits or loses

Bound queues, shed optional work, reserve critical capacity, and return a deliberate response. An unbounded queue converts overload into memory pressure and unusable tail latency.

Recovery

Design the replay path

Record durable intent, make consumers duplicate-tolerant, quarantine poison work, reconcile derived views, and prove that failover cannot expose two owners without fencing.

Evidence

Connect claim to signal

For each major decision name an SLI, load or fault test, saturation threshold, alert, rollback signal, and capacity owner. “Highly available” is not measurable until the request contract is defined.

Primary trailLittle’s Law paper ↗RFC 9110 HTTP semantics ↗RFC 9111 caching ↗Kafka delivery semantics ↗AWS transactional outbox ↗AWS retries + jitter ↗Google SRE availability + error budgets ↗Google SRE cascading failures ↗PostgreSQL multicolumn indexes ↗Cassandra data model ↗Consistent hashing paper ↗Chubby + fencing context ↗

11 · Interview practice arena

Practice the conversation—not a memorized diagram

Choose a prompt, work through each checkpoint aloud, and reveal the coaching notes only after committing to an answer.

Interview prompt

Design a URL shortening service

Users submit a long URL and receive a short link. Opening the short link redirects quickly and reliably.

    1. 00–05 minClarify the contract

      Ask questions before drawing. Separate core requirements from optional features.

      Your turnWhat traffic, retention, custom-alias, expiration, and analytics requirements matter?
      Reveal coaching notes
    2. 05–10 minEstimate the load

      Use round numbers to expose the likely bottleneck and storage shape.

      Your turnEstimate average/peak reads, writes, storage per year, and cacheable hot traffic.
      Reveal a worked estimate
    3. 10–20 minDefine API + data

      Name the contract and source of truth before adding infrastructure.

      Your turnDefine create and redirect APIs, then choose the minimum durable record.
      Reveal one reasonable model
    4. 20–32 minDraw the main path

      Explain one write and one read end-to-end. Keep the first design simple.

      Your turnWhich components handle creation, lookup, caching, persistence, and background work?
      Reveal a high-level flow
    5. 32–42 minDeep-dive the risk

      Let the workload choose the deep dive: hot keys, ordering, fan-out, delivery, or consistency.

      Your turnHow are unique short codes generated without collisions or a central bottleneck?
      Reveal trade-offs
    6. 42–45 minClose like an owner

      Summarize decisions, failure behavior, observability, security, and the next scaling step.

      Your turnWhat fails first, how do users experience it, and what do you monitor?
      Reveal closing checklist

    Follow-up pressure test

    Can your design survive changing requirements?

    10× traffic

    Identify the first saturated resource. Scale that component—not everything.

    One region fails

    State failover behavior, recovery objective, and consistency impact.

    One celebrity / hot key

    Prevent a single key, partition, or user from concentrating load.

    Duplicate requests

    Define idempotency keys, deduplication window, and safe retries.

    Dependency slows down

    Use deadlines, bounded retries, circuit breaking, and graceful degradation.

    Abuse begins

    Add authentication, quotas, rate limiting, validation, and audit signals.

    Self-score after speaking

    A strong answer makes reasoning visible

    • RequirementsDid I clarify scope and priorities?
    • NumbersDid estimates influence the design?
    • FlowCould someone trace reads and writes?
    • Trade-offsDid I explain why—not only what?
    • FailureDid I design degraded behavior?
    • CommunicationDid I confirm direction and manage time?

    12 · Deliberate study path

    Read in layers, then design from memory

    Use interview material for structure, foundational books for depth, and open repositories for repetition.

    01 · Begin

    System Design Interview, Vol. 1

    Framework, estimation, and approachable designs such as rate limiting, URL shortening, feeds, chat, and storage.

    Official overview ↗
    02 · Practice

    System Design Primer

    Free topic summaries, trade-offs, interview questions, sample solutions, diagrams, and Anki decks.

    GitHub repository ↗
    03 · Deepen

    Designing Data-Intensive Applications, 2nd ed.

    Updated coverage of reliability, scalability, data models, storage, replication, partitioning, transactions, streams, and modern distributed-data practice.

    Official 2nd-edition page ↗
    04 · Operate

    Google SRE books

    Availability, latency, capacity, monitoring, incident response, error budgets, and reliable production practice.

    Read online ↗
    05 · Expand

    Awesome System Design Resources

    A broad free index spanning fundamentals, networking, APIs, databases, distributed systems, and case studies.

    GitHub repository ↗
    06 · Advance

    System Design Interview, Vol. 2

    Deeper bottlenecks and trade-offs across queues, metrics, payments, object storage, maps, email, and exchanges.

    Official overview ↗
    07 · Protocol contract

    HTTP Semantics and Caching

    The IETF standards for methods, status codes, validators, conditional requests, freshness, shared caches, and cache-control behavior.

    RFC 9110 semantics ↗RFC 9111 caching ↗
    08 · Queueing

    A Proof for L = λW

    John Little’s foundational result connecting average arrival rate, time in a boundary, and average work in that boundary.

    INFORMS paper ↗
    09 · Consensus

    In Search of an Understandable Consensus Algorithm

    The extended Raft paper covering leader election, replicated logs, safety, membership changes, and the assumptions behind a consensus group.

    Raft paper ↗
    10 · Global data

    Spanner

    Google’s paper on synchronous replication, external consistency, automatic sharding, and explicit clock uncertainty at global scale.

    Google Research paper ↗
    11 · Failure control

    Timeouts, Retries, and Backoff with Jitter

    Operational guidance on timeout selection, retry amplification, token budgets, exponential backoff, and breaking correlated retry storms.

    AWS Builders’ Library ↗
    Weekly loopStudy one concept→Redraw it without notes→Design one product→List three trade-offs→Review after 1, 7, and 30 days