A cascading failure is one that grows over time as a result of positive feedback. One task gets overloaded and dies, its load moves to others, which then overload and die, until the whole service is down and cannot come back up because any task that starts is immediately crushed. This chapter is the book's longest technical chapter and probably the most important for system design interviews, because it explains the mechanism and then lists every tool for breaking the feedback loop.

The Shakespeare example runs through it: a datacenter drops out, its traffic redistributes, the remaining datacenters are each pushed over capacity, and the whole service fails.

In one minute

• Causes: server overload (losing capacity pushes the rest past their limit) and resource exhaustion (CPU, memory, threads, file descriptors, dependencies), each of which can push a server into a state where it fails faster than it serves.
• CPU exhaustion causes more in-flight requests, longer queues, slower GC, missed health checks, and timeouts that cause retries. Memory exhaustion causes task death, higher cache miss rates and more load upstream. Thread starvation and file descriptor exhaustion stop a server from accepting work at all.
• Service unavailability can be self-sustaining: a crashed server comes back, is instantly overloaded by backed-up clients and retries, and crashes again.
• Prevention: avoid overload via load testing to find the breaking point and provisioning with headroom, serve degraded results, reject requests when overloaded (chapter 21), use deadlines everywhere and propagate them, bound queue length (queue no more than about the number of worker threads), limit retries and use exponential backoff with jitter, and avoid retrying at multiple layers.
• Watch for latency and deadlines: a client whose deadline is longer than the server's work is wasting server time on results nobody will read. Propagate the remaining deadline, not the original one.
• Bimodal latency: when a dependency is slow for a fraction of requests, the slow fraction can consume all threads. Use timeouts and limit concurrency per dependency.
• Slow startup and cold caching turn a restart into an overload: provision for cold starts, warm caches, or ramp traffic slowly.
• Always go downward in the stack: a frontend should not call a backend that is already saturated more aggressively than before.
• Immediate steps: increase resources, stop health check failures and deaths (relax health checks that kill busy-but-alive tasks), restart servers (only if they are stuck, and not if they are just overloaded), drop traffic, enter degraded modes, eliminate batch load, eliminate bad traffic.
• Test: load test to failure to see how the service behaves past capacity, test popular clients, test non-critical backends, and test slow dependencies.

Causes of cascading failures and designing to avoid them

Server overload. Two clusters serve 10,000 QPS each with a capacity of 12,000. Cluster A fails; cluster B now sees 20,000 QPS, far over its limit, and its latency climbs until requests time out and fail. In a naive system all tasks in B then die, and because A's tasks are also trying to restart into the same flood, nothing recovers until traffic is cut. Overload is the usual starting point; everything else is about what happens after.

Resource exhaustion. CPU overload leads to longer queues, longer latency, more concurrent requests, more memory, more GC pressure, missed health checks, and, as timeouts expire, retries from clients. Memory exhaustion leads to tasks being killed by the container manager, cache evictions, and higher load on backends. Threads can be exhausted when a dependency is slow, leaving none to accept connections. File descriptors can run out from too many connections. Dependencies get starved when a client cascades its own load onto them. Each of these is a different path from "busy" to "broken".

Service unavailability. Once tasks crash, their load goes to the survivors. New tasks starting up have cold caches and less capacity, so they crash on arrival. A cluster can stay down even after the trigger is gone. The way out is to shed load until the servers can catch up.

Preventing server overload

Load test the server's capacity and the failure mode. Know the QPS at which latency starts climbing and the point at which the server dies, and provision so that losing a datacenter does not cross the first line. Serve degraded results when overloaded: a search can return results from a smaller index, a feed can skip personalisation. Build servers to reject requests when overloaded (chapter 21) rather than queue them. Have higher layers reject requests too: the load balancer and the frontend can drop traffic before it reaches the backend. Capacity planning is the long-term fix but will not save you today.

Queue management. A long queue is a lie: requests in it look accepted but will time out. Keep the queue short relative to the thread pool, around the number of threads or less, so that under overload the server fails fast. Use a controlled queueing algorithm such as CoDel if the service genuinely needs a queue.

Load shedding and graceful degradation. Shed by criticality (chapter 21), or by random sampling as a last resort. Degraded modes should be designed up front and tested regularly, because an untested degraded mode is a second outage waiting to happen.

Retries. Retries multiply load. Limit retries per request, apply a per-client retry budget, use randomised exponential backoff, and retry only at one layer. Consider whether a failed request should be retried at all: for a backend that is overloaded, a retry is just more load.

Latency and deadlines. Every RPC should have a deadline. Pick deadlines that reflect what the caller will actually wait for, and propagate the remaining time down the stack so a backend does not spend time computing a result whose caller has already given up. Check the deadline at each stage and abandon work that cannot finish in time. Beware bimodal latency: if 5 percent of calls to a dependency hang until timeout, they can hold 5 percent of all threads forever, which at high QPS is all of them.

Slow startup and cold caching

A freshly started task is slower than a warm one: JIT compilation, lazy initialisation, empty caches. If the service relies on caches to meet its capacity, a mass restart or a datacenter failover means serving at a fraction of capacity. Mitigations: over-provision so the service can survive on cold caches, warm the caches before accepting traffic, or ramp up traffic to new tasks gradually. In the Shakespeare example, the backends relied on a hot cache of the index, which is exactly what made the recovery fail.

Always go downward in the stack

A design where a backend calls back into its frontend, or where lateral communication between tasks is required, creates feedback loops that are hard to reason about during overload. Keep communication going down the stack from frontends to backends so that load flows one way.

Triggering conditions for cascading failures

Process death (from bugs, OOMs, or an over-eager health checker), process updates and rolling restarts, new rollouts that change the performance profile, organic growth that silently uses up the headroom, and planned changes such as turning down a datacenter can all tip a service over. Dependencies that fail or slow down are also common triggers; a service is only as robust as its weakest required dependency.

Testing for cascading failures

Load test to the breaking point and beyond, and see whether the service recovers when load is removed. Test the most popular clients' behaviour under failure, because their retries define the second wave. Test non-critical backends to be sure their absence is actually non-critical. Test what happens when a dependency is slow, not just down.

Immediate steps to address cascading failures

Add capacity if the trigger was a capacity loss. Stop health checks from killing tasks that are busy but alive, since a restarted task is worse than a slow one. Restart servers only if they are truly stuck (deadlocked, leaking) and only after confirming that restarting will not just re-trigger the cascade. Drop traffic at the load balancer, hard, to let servers come up, then increase gradually. Switch into degraded modes. Stop batch jobs and other non-essential load. Block abusive or bad traffic at the edge.

Closing remarks

Cascading failures are hard to recover from and easy to prevent with up-front design. The chapter's summary is that a service should know its capacity, degrade gracefully past it, fail fast rather than queue, bound its retries, honour deadlines, and have a tested procedure for coming back from zero.