System Design
Load balancers, caches, queues and regions: build a system that holds the load
- Single server. Everything on one machine — Requests and responses, RPS, latency, the capacity limit of one server and a single point of failure.
- Separate database. Data on its own machine — Why stateless servers scale more easily and where the data should live.
- Load balancer. Sharing the load — Horizontal scaling: several servers behind one entry point, health checks and the N+1 rule.
- Cache. Fast answers from memory — Cache-aside, hit rate and TTL: taking load off the database.
- CDN. Static files close to users — Static and dynamic content, edge locations and cache headers: serving images and scripts closer to the user.
- Read replicas. Database copies for reads — Read replicas, replication lag, failover and why every write still goes to the primary.
- Queue. Heavy work goes to the background — Moving slow work out of the request: a queue, workers, queue growth, bursts and at-least-once delivery.
- Rate limiter. Protection from overload — A rate limiter in front of the load balancer: a fast 429 instead of overload, per-client limits, backpressure and retries without a storm.
- Sharding. Splitting the data — Splitting data across several databases: the shard key, hot shards, cross-shard queries and resharding.
- Failover. Surviving a failure — Single points of failure: a second load balancer, a read replica that takes over the database, health checks, failover, and availability versus durability.
- Multi-region and CAP. When a whole region goes down — A second region for distant users and for the loss of a data center, asynchronous replication and the choice between consistency and availability.
- Search index and event bus. Search and events — An event bus delivers every change to the search index and to analytics: publish/subscribe, subscriber lag and eventual consistency.
- URL shortener. Final project — Design a whole system from scratch: requirements, estimates, API, then a design that holds the peak, survives failures and fits the budget.
- Public API. Limits and quotas — An API for developers with Free and Pro tiers: a core with headroom within budget, and when a Free key starts downloading the whole database, deciding whom to limit and whom to pay for.
- Flash sale. Concert tickets — Concert tickets sell out in minutes: a waiting room admits as many fans as the hot row can take, a conditional UPDATE and holds with a TTL prevent overselling, and a second date gets its own row and its own queue.
- Notification system. Codes and campaigns — A shop’s notifications: OTP codes and promotional campaigns take separate paths (a bulkhead), so Black Friday does not hold up sign-ins, and a spare provider saves the codes when one goes down.
- Logging pipeline. Buffer and sampling — Collecting the logs of 3,000 services: a buffer so agents lose nothing when the index falls behind, and sampling when the storm outlasts the buffer.
- News feed. Fan-out on write and the hybrid — A social network feed for 5 million DAU: ready-made timelines in a cache, fan-out through a queue, and when celebrities arrive, a hybrid that also saves money.
- Chat. A messenger for teams — A team chat in Europe and the USA: delivering events through gateways, a service in two regions with one write leader, message order, and what is lost when a region goes down.
- Video. Upload and streaming — A video platform for online courses: bytes go past the servers (uploads by presigned URL into object storage, viewing through a CDN), and transcoding runs in the background on workers sized for an ordinary evening, not for the peak.
- File sync. Files on every device — File sync for teams: metadata apart from content, chunks in object storage, changes through an event bus with notify-then-pull, and after an outage, clients coming back without a cascade.
- Ride matching. Taxis in two regions — A taxi service in Europe and the USA: driver locations bypass the database, each region has its own stream (geo-partitioning), and on New Year’s Eve Europe is split into cells.
- Payments. A payment gateway — A payment gateway for shops: never charge twice (idempotency key), never lose a confirmed payment (RPO = 0, transactional outbox), and survive losing a region.