System design concepts

The building blocks behind system-design answers, explained well enough to reason with. For each block you get what it is, when to reach for it, what it costs, and one sentence to say in an interview. The production reference with numbers, caching tables, queues, rate limiting and a worked URL shortener is System design; data modeling is in Database design. How to run the interview itself, with nine worked designs, is in System design interviews.

Networking & protocols

Server-to-client updates

Say it like this

• DNS: "DNS failover is bounded by the TTL, so for fast regional failover I'd put anycast or a global load balancer in front."
• TCP vs UDP: "It's live audio, so late packets are useless; UDP (or WebRTC) beats TCP retransmits here."
• HTTP/3: "Mobile clients on lossy networks benefit from HTTP/3, because a lost packet only stalls its own stream."
• SSE vs WebSockets: "Updates only flow server to client, so SSE over plain HTTP is enough; WebSockets only if the client streams too."
• Webhooks: "We deliver webhooks at least once with retries and a signature, and document that receivers must dedupe by event ID."

API design

Say it like this

• REST vs gRPC: "The public API is REST for cacheability and reach; internal calls are gRPC for typed contracts and deadlines."
• GraphQL: "Web and mobile need different shapes of the same data, so a GraphQL layer over the services saves round trips; I'd cap query cost."
• Cursor pagination: "Cursor pagination on (created_at, id) keeps deep pages an index seek and doesn't skip items when new posts arrive."
• Idempotency: "Clients retry payments on timeouts, so POST /payments takes an idempotency key and replays the stored result."

Load balancing & proxies

Full table with L4/L7 products and health-check details: System design: Scaling & load balancing.

Stateless services keep no user state in process memory: sessions go to Redis or a signed token, files to object storage, caches to a shared cache. Then any instance can serve any request, instances can die or be added at will, and autoscaling works. State lives in a few purpose-built stores that you scale deliberately.

Say it like this

• L7: "An L7 balancer lets me route /api and /ws to different pools and do canary releases by header."
• Stateless: "The API tier is stateless, sessions live in Redis, so I can scale it horizontally behind the balancer."
• Gateway: "Auth and per-key rate limits sit in the gateway so every service doesn't reimplement them."
• Discovery: "Instances register with the service registry and the balancer only routes to ones passing readiness checks."

Caching

A cache trades freshness for speed. Effective latency is $h \cdot t_{cache} + (1 - h) \cdot t_{db}$: at a 95% hit rate, 1 ms cache and 10 ms database, reads average 1.45 ms. Every percent of hit rate you gain removes load from the database, which is usually the real goal.

Redis picks victims only when maxmemory is reached and the default policy is noeviction (writes fail), so a cache needs allkeys-lru or allkeys-lfu set explicitly (Redis eviction docs (opens in a new tab)).

More fixes and a single-flight snippet: System design: Caching & CDNs.

Say it like this

• Cache-aside: "Reads go cache-aside with a TTL; on write we update Postgres, then delete the key, so the next read reloads it."
• Why a cache: "The read:write ratio is 100:1 and the data tolerates a few seconds of staleness, so a cache takes most load off the database."
• Stampede: "To stop a stampede on the celebrity profile, one request refreshes the key while the others get the stale value."
• CDN: "Video segments and thumbnails are immutable, so the CDN caches them forever under content-hashed names."

Data stores

SQL vs NoSQL is really "general-purpose queries vs design-for-known-access-patterns". Relational databases let you ask new questions later and keep invariants with transactions; NoSQL stores make you model tables around the queries you already know, and in return partition and replicate automatically. Distributed SQL (Spanner, CockroachDB) narrows the gap at the cost of write latency. Default to Postgres and name the specific access pattern that pushes you elsewhere.

Indexes: B-tree vs LSM-tree

Schema, normalization and composite-index design: Database design.

Say it like this

• Default: "I'll start with Postgres: the data is relational and we need transactions; one primary with replicas covers this scale."
• Wide-column: "Messages are append-heavy and always read by conversation, so Cassandra with conversation_id as the partition key and time as the clustering key fits."
• LSM: "This is write-heavy telemetry, so an LSM-based store absorbs writes as sequential appends."
• Search: "Postgres stays the source of truth; Elasticsearch is a projection fed by CDC, so it can be rebuilt."

Scaling data

Read-your-writes on async replicas: after a user writes, read their data from the leader for a few seconds, or send the write's log position and have the replica wait until it has caught up.

Sharding strategies

Detail and shard-key advice: System design: Sharding.

Consistent hashing

Hash servers and keys onto the same ring (0 … 2³² − 1). A key belongs to the first server point clockwise from it. Adding a server takes over only the arcs just before its points, about 1/N of keys; with hash mod N almost every key would move. Virtual nodes (each server at 100–200 points) even out the arcs, spread a dead server's load across all survivors, and let a bigger machine take more points.

// FNV-1a over UTF-8 bytes, then a murmur3 finalizer
// so near-identical strings land far apart
function hash32(s: string, seed = 0x811c9dc5): number {
  let h = seed;
  for (const b of new TextEncoder().encode(s)) {
    h = Math.imul(h ^ b, 0x01000193);
  }
  h = Math.imul(h ^ (h >>> 16), 0x85ebca6b);
  h = Math.imul(h ^ (h >>> 13), 0xc2b2ae35);
  return (h ^ (h >>> 16)) >>> 0;
}
 
type VNode = { pos: number; node: string };
 
class HashRing {
  private ring: VNode[] = [];
 
  constructor(private readonly vnodes = 100) {}
 
  add(node: string): void {
    for (let i = 0; i < this.vnodes; i++) {
      const pos = hash32(`${node}#${i}`);
      this.ring.push({ pos, node });
    }
    this.ring.sort((a, b) => a.pos - b.pos);
  }
 
  remove(node: string): void {
    this.ring = this.ring.filter((v) => v.node !== node);
  }
 
  nodeFor(key: string): string {
    if (this.ring.length === 0) throw new Error("empty");
    const h = hash32(key);
    let lo = 0;
    let hi = this.ring.length; // first pos ≥ h
    while (lo < hi) {
      const mid = (lo + hi) >>> 1;
      if (this.ring[mid].pos < h) lo = mid + 1;
      else hi = mid;
    }
    return this.ring[lo % this.ring.length].node; // wrap
  }
}

// FNV-1a over UTF-8 bytes, then a murmur3 finalizer
// so near-identical strings land far apart
function hash32(s, seed = 0x811c9dc5) {
  let h = seed;
  for (const b of new TextEncoder().encode(s)) {
    h = Math.imul(h ^ b, 0x01000193);
  }
  h = Math.imul(h ^ (h >>> 16), 0x85ebca6b);
  h = Math.imul(h ^ (h >>> 13), 0xc2b2ae35);
  return (h ^ (h >>> 16)) >>> 0;
}
 
class HashRing {
  #ring = [];
  #vnodes;
 
  constructor(vnodes = 100) {
    this.#vnodes = vnodes;
  }
 
  add(node) {
    for (let i = 0; i < this.#vnodes; i++) {
      const pos = hash32(`${node}#${i}`);
      this.#ring.push({ pos, node });
    }
    this.#ring.sort((a, b) => a.pos - b.pos);
  }
 
  remove(node) {
    this.#ring = this.#ring.filter((v) => v.node !== node);
  }
 
  nodeFor(key) {
    if (this.#ring.length === 0) throw new Error("empty");
    const h = hash32(key);
    let lo = 0;
    let hi = this.#ring.length; // first pos ≥ h
    while (lo < hi) {
      const mid = (lo + hi) >>> 1;
      if (this.#ring[mid].pos < h) lo = mid + 1;
      else hi = mid;
    }
    return this.#ring[lo % this.#ring.length].node; // wrap
  }
}

from bisect import bisect_left
 
M32 = 0xFFFFFFFF
 
 
def hash32(s: str, seed: int = 0x811C9DC5) -> int:
    h = seed
    for b in s.encode():
        h = ((h ^ b) * 0x01000193) & M32
    h = ((h ^ (h >> 16)) * 0x85EBCA6B) & M32
    h = ((h ^ (h >> 13)) * 0xC2B2AE35) & M32
    return h ^ (h >> 16)
 
 
class HashRing:
    def __init__(self, vnodes: int = 100) -> None:
        self.vnodes = vnodes
        self.ring: list[tuple[int, str]] = []  # (pos, node)
 
    def add(self, node: str) -> None:
        for i in range(self.vnodes):
            self.ring.append((hash32(f"{node}#{i}"), node))
        self.ring.sort()
 
    def remove(self, node: str) -> None:
        self.ring = [v for v in self.ring if v[1] != node]
 
    def node_for(self, key: str) -> str:
        if not self.ring:
            raise LookupError("empty")
        h = hash32(key)
        i = bisect_left(self.ring, h, key=lambda v: v[0])
        return self.ring[i % len(self.ring)][1]  # wrap

add: $O(V \log V)$ for $V$ virtual nodes in total (re-sort); nodeFor: $O(\log V)$; space $O(V)$. With 4 servers × 100 virtual nodes and 20,000 keys, adding a fifth server moved 25% of keys (ideal: 20%), all of them to the new server.

Hot partitions and resharding

Say it like this

• Replicas: "Reads outnumber writes 50 to 1, so I'll add read replicas and route read-after-write for the author to the leader."
• Shard key: "I'll shard by user_id: it's in every query, high-cardinality, and spreads writes evenly."
• Consistent hashing: "The cache tier uses consistent hashing with virtual nodes, so adding a node only moves about 1/N of the keys."
• Hot partition: "A celebrity's counter would melt one shard, so I split it into 10 sub-keys and sum on read."

Consistency & transactions

CAP, stated correctly: when a network partition separates replicas, a system must either refuse some requests (keep Consistency, meaning linearizability) or answer them with possibly stale data (keep Availability). Partitions aren't optional, so "pick two of three" is misleading: the real choice is what happens during a partition. PACELC adds the everyday case: if Partitioned, choose A or C; else choose Latency or Consistency, because waiting for replicas costs time even when nothing is broken.

Definitions in depth: Jepsen: consistency models (opens in a new tab).

Quorums

With $N$ replicas, a write waits for $W$ acknowledgments and a read asks $R$ replicas. If $R + W > N$, every read set overlaps every write set, so at least one replica in the read has the latest acknowledged write (pick it by version). $N = 3, W = 2, R = 2$ tolerates one slow or dead node for both reads and writes. $W = N, R = 1$ gives fast reads and fragile writes; $W = 1, R = N$ the reverse. Quorums alone are not linearizable: sloppy quorums (writes land on stand-in nodes during failures), concurrent writes and partially failed writes break the guarantee (DDIA ch. 5).

Transactions and isolation

ACID: Atomic (all or nothing), Consistent (invariants hold; mostly the app's job), Isolated (concurrent transactions don't see each other's partial work), Durable (committed data survives a crash).

Fixes without going serializable: atomic updates (SET n = n + 1), SELECT … FOR UPDATE, compare-and-set on a version column, unique constraints. Postgres behavior: transaction isolation docs (opens in a new tab).

Across services

Publishing events reliably alongside a DB write: the transactional outbox.

Say it like this

• CAP: "For the inventory counter I'd choose consistency during a partition: better to reject an order than oversell."
• PACELC: "Even without partitions, synchronous cross-region replication adds about 70 ms per write, so the feed stays eventually consistent."
• Quorum: "With N = 3, W = 2, R = 2 any read overlaps the last write and we survive one node down."
• Isolation: "Two bookings could race for the last seat, so I'd use SELECT … FOR UPDATE on the seat row, or a unique constraint on (flight, seat)."
• Saga: "Checkout is a saga: reserve stock, charge, ship; if charging fails, we release the reservation."

Async processing

Kafka vs RabbitMQ vs SQS side by side: System design: Queues & streams.

Fan-out on write vs on read

Twitter's timeline service used the hybrid: fan-out on write into cached timelines, with very large accounts merged in at read time (Timelines at Scale (opens in a new tab)).

Say it like this

• Queue: "Sending email doesn't need to block the request, so the API enqueues a job and returns 202."
• Kafka: "Order events go to Kafka keyed by order_id, so each order's events stay in order and analytics can replay them."
• Exactly once: "Delivery is at least once, so the consumer dedupes by event ID in the same transaction as its write."
• Fan-out: "Push to followers' feeds on write, except accounts over about 10k followers, whose posts get merged in at read time."

Coordination

Raft in one paragraph. Each node is a follower, candidate or leader. A follower that hears no heartbeat within a randomized election timeout (150–300 ms in the paper) becomes a candidate, increments the term and asks for votes; a node grants one vote per term, and only to candidates whose log is at least as up to date as its own. A majority makes a leader. The leader appends client commands to its log and replicates them; an entry is committed once stored on a majority, then applied to each node's state machine. Because any two majorities overlap, a committed entry survives every future leader change (Raft paper (opens in a new tab)). Use 3 nodes to survive 1 failure, 5 to survive 2.

Fencing, the classic bug: worker 1 takes a lease, pauses for GC longer than the lease, worker 2 takes the lock and writes, worker 1 wakes up and writes too. With fencing tokens worker 1 holds token 33, worker 2 holds 34, and storage refuses 33 after seeing 34 (Kleppmann: How to do distributed locking (opens in a new tab)).

Clocks

Unique IDs: use time-ordered IDs for index locality: Snowflake (64-bit: timestamp, worker, sequence), ULID and UUIDv7 (128-bit, millisecond-sortable). Layouts and trade-offs: System design: IDs.

Say it like this

• Consensus: "Leader election and config live in etcd, which uses Raft, so with 3 nodes we survive one failure without split-brain."
• Locks: "The lock has a 30-second lease and a fencing token, so a stale holder can't overwrite newer work."
• Clocks: "I won't order messages by server timestamps because clocks skew; the partition's sequence number defines order."
• IDs: "Message IDs are Snowflake-style: sortable by time, generated locally without coordination."

Specialized structures

Geohash cells cut through neighborhoods: two points 10 m apart can sit on either side of a boundary with no common prefix, so always search the cell and its 8 neighbors (Geohash precision table (opens in a new tab)).

Bloom filter

With $m$ bits, $k$ hash functions and $n$ items inserted, the false-positive rate is $p \approx (1 - e^{-kn/m})^k$. Sizing for $n$ items at a target $p$: $m = -\frac{n \ln p}{(\ln 2)^2}$ bits and $k = \frac{m}{n} \ln 2$ hash functions. The $k$ positions come from two hashes as $h_1 + i \cdot h_2$ (double hashing). Uses hash32 from the ring above.

class BloomFilter {
  private readonly bits: Uint8Array;
 
  constructor(
    private readonly m: number, // bits
    private readonly k: number, // hash functions
  ) {
    this.bits = new Uint8Array(Math.ceil(m / 8));
  }
 
  static forCapacity(n: number, p: number): BloomFilter {
    const m = Math.ceil((-n * Math.log(p)) / Math.LN2 ** 2);
    const k = Math.max(1, Math.round((m / n) * Math.LN2));
    return new BloomFilter(m, k);
  }
 
  private indexes(item: string): number[] {
    const h1 = hash32(item);
    const h2 = hash32(item, 0x9747b28c);
    return Array.from(
      { length: this.k },
      (_, i) => (h1 + i * h2) % this.m, // double hashing
    );
  }
 
  add(item: string): void {
    for (const i of this.indexes(item)) {
      this.bits[i >> 3] |= 1 << (i & 7);
    }
  }
 
  mightContain(item: string): boolean {
    return this.indexes(item).every(
      (i) => (this.bits[i >> 3] & (1 << (i & 7))) !== 0,
    );
  }
}

class BloomFilter {
  #bits;
  #m;
  #k;
 
  constructor(
    m, // bits
    k, // hash functions
  ) {
    this.#m = m;
    this.#k = k;
    this.#bits = new Uint8Array(Math.ceil(m / 8));
  }
 
  static forCapacity(n, p) {
    const m = Math.ceil((-n * Math.log(p)) / Math.LN2 ** 2);
    const k = Math.max(1, Math.round((m / n) * Math.LN2));
    return new BloomFilter(m, k);
  }
 
  #indexes(item) {
    const h1 = hash32(item);
    const h2 = hash32(item, 0x9747b28c);
    return Array.from(
      { length: this.#k },
      (_, i) => (h1 + i * h2) % this.#m, // double hashing
    );
  }
 
  add(item) {
    for (const i of this.#indexes(item)) {
      this.#bits[i >> 3] |= 1 << (i & 7);
    }
  }
 
  mightContain(item) {
    return this.#indexes(item).every(
      (i) => (this.#bits[i >> 3] & (1 << (i & 7))) !== 0,
    );
  }
}

import math
 
 
class BloomFilter:
    def __init__(self, m: int, k: int) -> None:
        self.m, self.k = m, k  # bits, hash functions
        self.bits = bytearray((m + 7) // 8)
 
    @classmethod
    def for_capacity(cls, n: int, p: float) -> "BloomFilter":
        m = math.ceil(-n * math.log(p) / math.log(2) ** 2)
        k = max(1, round(m / n * math.log(2)))
        return cls(m, k)
 
    def _indexes(self, item: str) -> list[int]:
        h1 = hash32(item)
        h2 = hash32(item, 0x9747B28C)
        return [(h1 + i * h2) % self.m
                for i in range(self.k)]
 
    def add(self, item: str) -> None:
        for i in self._indexes(item):
            self.bits[i >> 3] |= 1 << (i & 7)
 
    def might_contain(self, item: str) -> bool:
        return all(
            self.bits[i >> 3] & (1 << (i & 7))
            for i in self._indexes(item)
        )

add and mightContain: $O(k)$ time; space $m$ bits. For 1,000 items at 1%: 9,586 bits (1.2 KB) and 7 hashes; measured false-positive rate on 100,000 absent keys: 1.04%.

Say it like this

• Bloom filter: "Before hitting the database for 'is this username taken', a Bloom filter answers most 'no' cases from memory."
• HyperLogLog: "Unique viewers per video per day go into a HyperLogLog: 12 KB per counter, under 1% error, and they merge across days."
• Count-min: "Trending hashtags: a count-min sketch per minute window plus a min-heap of the top 100."
• Geo: "Businesses are indexed by geohash; a search covers the user's cell and the 8 neighbors, then filters by exact distance."
• Merkle: "Replicas compare Merkle trees in the background and only exchange the ranges whose hashes differ."

Reliability & availability

Computed as $(1 - A) \times$ period with a 365-day year. In series, availabilities multiply (two 99.9% hops give 99.8%); in parallel, redundant copies fail together only with probability $(1 - A)^n$ (two independent 99% replicas give 99.99%), but real failures are often correlated (same deploy, same region). See the SRE book on embracing risk (opens in a new tab).

Distributed versions, headers and a comparison: System design: Rate limiting.

Token bucket

class TokenBucket {
  private tokens: number;
  private last: number;
 
  constructor(
    private readonly capacity: number, // burst size
    private readonly perSec: number, // refill rate
    now: number = performance.now(),
  ) {
    this.tokens = capacity;
    this.last = now;
  }
 
  tryTake(cost = 1, now = performance.now()): boolean {
    const elapsed = (now - this.last) / 1000; // ms → s
    this.tokens = Math.min(
      this.capacity,
      this.tokens + elapsed * this.perSec,
    );
    this.last = now;
    if (this.tokens < cost) return false;
    this.tokens -= cost;
    return true;
  }
}

class TokenBucket {
  #tokens;
  #last;
  #capacity;
  #perSec;
 
  constructor(
    capacity, // burst size
    perSec, // refill rate
    now = performance.now(),
  ) {
    this.#capacity = capacity;
    this.#perSec = perSec;
    this.#tokens = capacity;
    this.#last = now;
  }
 
  tryTake(cost = 1, now = performance.now()) {
    const elapsed = (now - this.#last) / 1000; // ms → s
    this.#tokens = Math.min(
      this.#capacity,
      this.#tokens + elapsed * this.#perSec,
    );
    this.#last = now;
    if (this.#tokens < cost) return false;
    this.#tokens -= cost;
    return true;
  }
}

import time
 
 
class TokenBucket:
    def __init__(self, capacity: float, per_sec: float,
                 now: float | None = None) -> None:
        self.capacity = capacity  # burst size
        self.per_sec = per_sec  # refill rate
        self.tokens = capacity
        self.last = time.monotonic() if now is None else now
 
    def try_take(self, cost: float = 1,
                 now: float | None = None) -> bool:
        now = time.monotonic() if now is None else now
        elapsed = now - self.last  # already seconds
        self.tokens = min(
            self.capacity,
            self.tokens + elapsed * self.per_sec,
        )
        self.last = now
        if self.tokens < cost:
            return False
        self.tokens -= cost
        return True

$O(1)$ time and space per key: refill lazily from elapsed time instead of running a timer. Across many servers, keep tokens and last per key in Redis and update both in one Lua script so the read-modify-write is atomic. The now parameter makes it testable with a fake clock.

Say it like this

• Nines: "Four nines is about 52 minutes a year, so failover has to be automatic; a human can't be paged and fix it in time."
• Retries: "Retries use exponential backoff with full jitter and a retry budget, only at the edge, so we don't multiply load during an outage."
• Circuit breaker: "If the recommendations service trips its breaker, the page renders without recommendations instead of timing out."
• Load shedding: "Past 80% of capacity we shed anonymous traffic first and keep checkout working."
• Multi-region: "Each user has a home region for writes; reads are served locally and cross-region replication is async."

Batch vs stream processing

MapReduce: map turns each input record into key-value pairs, the framework shuffles so all values for a key reach the same reducer, reduce combines them. Failed tasks just rerun because inputs are immutable (MapReduce paper (opens in a new tab)).

Stream processing adds windows (tumbling, sliding, session), event time vs processing time, watermarks (how long to wait for late events) and checkpointed state so a restart resumes exactly where it left off.

Say it like this

• Batch vs stream: "Billing reports can be a nightly batch; fraud scoring must be streaming because it has to block the payment."
• Kappa: "I'd keep events in Kafka with long retention, so fixing a bug means replaying the topic through the new job."
• CDC: "The search index is fed by CDC from Postgres, so there's no dual write that can drift."

Security & observability

Dashboards and alerting detail: System design: Observability.

Say it like this

• AuthN vs authZ: "The gateway authenticates the JWT; each service still authorizes the action against the resource owner."
• JWT: "Access tokens live 15 minutes; revocation happens at refresh, which checks the session store."
• SLO: "Our SLO is 99.9% of feed loads under 500 ms; alerts fire on error-budget burn rate, not on single spikes."

Trade-off → phrase

Recipes

"Which database?" checklist

Walk these in order and say each answer out loud.

• What are the top 3 queries, and by which key? Point lookups, ranges, full text, graph hops, nearest neighbor?
• Do you need multi-row transactions or constraints (money, inventory, uniqueness)? → relational.
• Reads and writes per second at peak, and data size in 3 years? Does it fit one primary with replicas?
• Is the shape a self-contained aggregate (document), append-only by key and time (wide-column, time-series), or files (object store)?
• Which consistency does each query need (see Consistency & transactions)?
• Default: Postgres; add Redis (cache), an object store (files) and a search index (projection) as named needs appear.

"Make it faster" checklist

• Measure first: which endpoint, p50 or p99, where does the time go (trace)?
• Cache the hot reads (cache-aside, CDN for static and immutable data).
• Add the missing index; fix N+1 queries; paginate with cursors.
• Precompute: denormalize, materialized views, fan-out on write.
• Move work off the request path: queue it and return 202.
• Cut round trips: batch requests, HTTP/2 or HTTP/3, keep connections alive, colocate services with data.
• Serve from closer: CDN, edge compute, read replicas per region.
• Scale out the stateless tier; then read replicas; shard last.

"Make it reliable" checklist

• Remove single points of failure: N+1 instances across 3 zones, replicated data, automatic failover.
• Timeouts on every call, retries with backoff and jitter only for idempotent operations, idempotency keys.
• Circuit breakers and bulkheads around each dependency; a fallback for each.
• Queues between tiers to absorb spikes; dead-letter queues for poison messages.
• Rate limiting at the edge; load shedding under overload.
• Backups you have actually restored; defined RPO (data loss allowed) and RTO (downtime allowed).
• SLOs with burn-rate alerts; health checks separating liveness from readiness; practiced failover drills.

Pick a consistency level per operation

• Money, inventory, uniqueness, locks → linearizable (single leader, consensus, or SERIALIZABLE).
• A user's own recent writes → read-your-writes (read from leader briefly, or wait for replica position).
• Conversations, comment threads → causal (per-conversation sequence numbers).
• Counts, likes, feeds, analytics → eventual; show "about" numbers.

Explain any building block in 30 seconds

1. What it is, in one sentence.
2. Why this system needs it (tie to a requirement or number).
3. What it costs and what you'll do about it.
4. What you'd monitor to know it's working (hit rate, lag, error rate).

Example: "A Redis cache in front of profiles. Reads are 95% of traffic and profiles change rarely, so a 5-minute TTL takes most load off Postgres. The cost is staleness, so we delete the key on update. I'd watch hit rate and p99."

Scale a hot write path

• Batch: buffer writes and flush every N ms or N items.
• Make writes append-only (log, LSM store) instead of updates in place.
• Split hot keys into sub-keys; aggregate on read or in a periodic job.
• Queue writes and let workers drain at the database's pace.
• Shard by a key that spreads writes; keep related rows on the same shard.

References

• Amazon Dynamo paper (2007) (opens in a new tab): consistent hashing, vnodes, quorums, hinted handoff, Merkle trees
• Raft paper (opens in a new tab) and raft.github.io (opens in a new tab): consensus with a visualization
• Google Bigtable paper (opens in a new tab): wide-column model, SSTables
• MapReduce paper (opens in a new tab): the batch model
• Scaling Memcache at Facebook (opens in a new tab): leases against stampedes and stale sets
• RFC 9110: HTTP semantics (opens in a new tab), RFC 9114: HTTP/3 (opens in a new tab), RFC 8446: TLS 1.3 (opens in a new tab), RFC 6455: WebSocket (opens in a new tab): protocol facts
• MDN: Server-sent events (opens in a new tab): EventSource, reconnection, connection limits
• PostgreSQL: transaction isolation (opens in a new tab): what each level prevents in Postgres
• Redis: HyperLogLog (opens in a new tab) and key eviction (opens in a new tab): sizes, error, eviction policies
• Kleppmann: How to do distributed locking (opens in a new tab): leases and fencing tokens
• Jepsen: consistency models (opens in a new tab): precise definitions
• Designing Data-Intensive Applications (opens in a new tab): the book behind most of this sheet (replication, partitioning, transactions, streams)
• The System Design Primer (opens in a new tab): broad index of the same topics
• Google SRE book (opens in a new tab): availability, SLOs, overload handling
• AWS: exponential backoff and jitter (opens in a new tab): why full jitter
• Twitter: Timelines at Scale (opens in a new tab): the hybrid fan-out in production
• System design (production reference): numbers, tables and the worked URL shortener