Most applications today are data-intensive rather than compute-intensive: the hard problems are the volume of data, its complexity, and how fast it changes, not raw CPU. Such applications are assembled from a handful of standard building blocks (databases, caches, search indexes, stream processors, batch processors), and the interesting engineering happens at the seams, where you glue those blocks together behind an API and have to guarantee something about the whole.

This chapter sets up the vocabulary for the rest of the book. It asks three questions of any data system: does it keep working correctly when things go wrong (reliability), does it have sensible options for coping with growth (scalability), and can people keep working on it productively over the years (maintainability)? Interviewers lean on this vocabulary constantly, so being precise about faults versus failures, percentiles versus averages, and scaling up versus out pays off immediately.

In one minute

• A fault is one component deviating from its spec; a failure is the system as a whole not delivering the service the user expects. Fault tolerance means preventing faults from turning into failures, not preventing faults.
• Hardware faults are random and mostly independent; software bugs are systematic and correlated (one bad input or one leap second can take down every node at once); human error is the leading cause of outages in practice.
• Scalability is not a yes/no property. You describe load parameters (requests per second, read/write ratio, fan-out, active users), then ask how performance changes as a parameter grows.
• Report latency as percentiles (p50, p95, p99, p999), never as a mean. The slow tail is what your most valuable users experience, and tail latency amplification means one slow backend call out of many makes the whole page slow.
• Measure response time from the client, with a load generator that keeps sending requests independently of how slow the server is, or queueing effects will hide the problem.
• Scaling up (a bigger machine) is simpler; scaling out (a shared-nothing cluster of smaller machines) is the only route past a certain point and introduces the distributed-systems problems the rest of the book is about.
• There is no generic scalable architecture: the design that fits your load parameters is the one that scales.
• Maintainability decomposes into operability (easy to run), simplicity (easy to understand, mostly by removing accidental complexity through good abstractions), and evolvability (easy to change).

Thinking about data systems

Why "data system" and not "database"

The traditional categories have blurred. Redis is a datastore that is also used as a message queue; Kafka is a message queue with database-like durability; PostgreSQL can do full-text search; Elasticsearch is a search index that people treat as a primary store (usually unwisely). Because no single tool meets every requirement, a typical application stitches several together: a relational database as the source of truth, a cache in front of it, a search index fed asynchronously, and a queue that feeds background workers.

Once you compose tools like that, you have built a special-purpose data system, and you inherit the questions a database designer faces. How do you keep the cache and the search index consistent with the primary store? What happens to in-flight requests when one component is degraded? What are the guarantees your API makes to its callers? Application developers are therefore also data-system designers, whether or not they chose to be.

The three concerns

Kleppmann frames the chapter around three properties:

Concern	Question it answers	Typical failure to think about it
Reliability	Does the system keep doing the right thing when hardware, software or people misbehave?	A single disk failure or a bad deploy causes a user-visible outage.
Scalability	As load grows, are there reasonable ways to keep performance acceptable?	The architecture that worked at 1,000 users collapses at 1,000,000.
Maintainability	Can engineers and operators work on the system productively over many years?	Every change is slow and risky because nobody understands the whole thing.

Reliability

Faults versus failures

Informally, reliable software does what the user expects, tolerates mistakes and surprising usage, performs well enough under the expected load, and prevents abuse and unauthorized access. The book sharpens this with two terms.

A fault is one component of the system deviating from its specification: a disk returns a read error, a process throws an exception, an operator fat-fingers a config value. A failure is the system as a whole stopping to provide the required service to the user. A fault-tolerant (or resilient) system is designed so that faults do not become failures.

Two consequences follow. First, you cannot drive the fault rate to zero, so the design goal is containment, not prevention. Second, since you expect faults, you should provoke them deliberately: kill processes at random (Netflix's Chaos Monkey is the canonical example), pull network cables in staging, fill up disks. A fault-tolerance mechanism that has never been exercised is a mechanism you do not know works.

There are cases where prevention beats tolerance, chiefly security: once an attacker has exfiltrated data, no amount of tolerance undoes it. But the book, and this chapter, is about faults that can be tolerated.

Hardware faults

Disks crash, RAM goes bad, power grids black out, someone unplugs the wrong cable. The textbook number is that hard disks have a mean time to failure of roughly 10 to 50 years, so in a storage cluster of 10,000 disks you should expect about one disk to die every day. Hardware faults are typically random and independent of each other: one machine's disk failing does not make another's more likely (barring shared causes such as temperature in a rack).

The classic response is hardware redundancy: RAID for disks, dual power supplies, hot-swappable CPUs, generators and batteries in the datacenter. For a long time that was enough, because a whole-machine failure was rare and restoring from backup was acceptable.

Two trends have pushed systems toward software fault tolerance in addition to, or instead of, hardware redundancy. Data volumes and compute demands have grown, so applications use many more machines, which proportionally increases the rate of hardware faults. And cloud platforms such as AWS are designed to prioritize flexibility and elasticity over single-machine reliability, so virtual machine instances become unavailable without warning. A system that can tolerate the loss of an entire machine has operational advantages too: you can patch one node at a time (a rolling upgrade) instead of scheduling downtime for the whole system.

Software errors

Software faults are harder than hardware faults because they are systematic and correlated across nodes. The same bug exists on every replica, so a trigger that affects one node tends to affect all of them simultaneously. Examples from the book and from practice:

• A bug that crashes every instance of an application server when given a particular bad input. The leap second on 30 June 2012 made many Linux kernels hang at once because of a bug in the kernel's handling of the leap second.
• A runaway process that uses up a shared resource (CPU, memory, disk space, network bandwidth) and starves everything else on the box.
• A service the system depends on that slows down, becomes unresponsive, or starts returning corrupted responses.
• Cascading failures, where a small fault in one component triggers a fault in another, which triggers a fault in a third. A classic shape: one database replica slows down, clients time out and retry, the retries push the replica over the edge, the load shifts to the remaining replicas, and they fall over too.

These bugs usually lie dormant until an assumption about the environment stops holding. There is no quick fix, only a collection of habits: think carefully about assumptions and interactions, test thoroughly, isolate processes so one cannot take down another, allow processes to crash and restart, and measure, monitor and analyze behavior in production. If a system is supposed to maintain some invariant (say, the number of messages leaving a queue equals the number entering it), it can continually check that invariant and raise an alert when it is violated.

Human errors

Humans design, build and operate systems, and humans are unreliable. One study of large internet services found that configuration errors by operators were the leading cause of outages, whereas hardware faults caused only 10 to 25 percent. Good systems combine several approaches:

• Design interfaces that make the right thing easy and the wrong thing hard. Well-designed abstractions, APIs and admin interfaces discourage misuse. The balance is delicate: if an interface is too restrictive, people work around it, and the workaround is worse than what you prevented.
• Decouple the places where people make mistakes from the places where mistakes cause failures. Provide full-featured, non-production sandbox environments where people can explore and experiment with real data, without affecting real users.
• Test thoroughly at all levels: unit tests, whole-system integration tests, manual tests. Automated testing is especially valuable for corner cases that rarely arise in normal operation.
• Make recovery fast and easy. Roll back configuration changes quickly, roll out new code gradually so a bug affects a small subset of users, and provide tools to recompute data when an old computation turns out to have been wrong.
• Set up detailed and clear monitoring, covering performance metrics and error rates. The book calls this telemetry, after the instrumentation on rockets. It gives early warning and lets you diagnose after the fact.
• Implement good management practices and training, which the book notes is important but outside its scope.

How much reliability is enough

Reliability matters beyond nuclear plants and air traffic control. An outage of an e-commerce site costs revenue and reputation; a bug that corrupts a user's photo library destroys something irreplaceable. There are situations where you deliberately trade reliability for development cost or operational cost (a prototype, a service with tiny margins), but that should be a conscious decision, not an accident.

Scalability

Scalability is a question, not an adjective

A system that works today may not work with ten times the users or ten times the data. Scalability is the term for a system's ability to cope with increased load, but the book insists it is meaningless as a one-dimensional label ("X is scalable"). The useful questions are: if the system grows in a particular way, what are the options for coping? How can we add computing resources to handle the extra load?

Describing load

You cannot discuss growth until you can describe current load. Load parameters are the numbers that best characterize what the system is doing. Which ones matter depends on the architecture: requests per second to a web server, ratio of reads to writes in a database, number of simultaneously active users in a chat room, hit rate on a cache, fan-out of a social graph. Sometimes the average case matters; often it is a small number of extreme cases that dominate.

The book's running example is Twitter circa 2012, with two main operations:

• Post tweet: about 4,600 requests per second on average, over 12,000 per second at peak.
• Home timeline: a user views tweets from the people they follow, about 300,000 requests per second.

Handling 12,000 writes per second is easy. The scaling challenge is fan-out: each user follows many people and is followed by many people. Two implementations are possible.

Approach 1, join on read. Store tweets in a global table. To build a home timeline, look up everyone the user follows, find all their tweets, and merge sorted by time. In SQL this is a three-way join between users, follows and tweets. Writes are trivial; reads do the work.

Approach 2, fan-out on write. Maintain a cache (a per-user mailbox) of each user's home timeline. When someone tweets, look up all their followers and insert the tweet into each follower's cache. Reading a timeline is then a single cheap lookup; the work has moved to write time.

Twitter's first version used approach 1 and could not keep up with the home-timeline query load, so it moved to approach 2. The arithmetic explains why: the average tweet is delivered to about 75 followers, so 4,600 tweets per second become about 345,000 writes per second to timeline caches. That is manageable, and it makes the 300,000 reads per second nearly free. But the average hides the hard case. Some users have over 30 million followers, so a single tweet from them means 30 million cache writes, and Twitter's goal of delivering tweets within about five seconds makes that a serious problem.

So the decisive load parameter for Twitter is the distribution of followers per user, weighted by how often those users tweet, not the raw tweet rate. Twitter's eventual design is a hybrid: most tweets fan out on write, but tweets from a small number of celebrities are excluded from the fan-out and are fetched separately and merged at read time. This pattern (fast path for the common case, special handling for the heavy tail) recurs throughout system design interviews.

Describing performance

With load described, you can investigate performance in two ways: hold resources fixed and increase a load parameter, and watch what happens; or increase a load parameter and ask how much you must add to keep performance unchanged. Both need a performance number.

In a batch processing system such as Hadoop, the number is throughput: records processed per second, or total time to run a job over a dataset of a given size. In an online system, the number is response time: what the client sees between sending a request and receiving the response. (The book distinguishes latency, the time a request spends waiting to be handled, from response time, which also includes service time and network delays, though in casual use the two words are swapped freely.)

Response time is not a single number but a distribution. The same request repeated many times varies because of context switches, lost packets and TCP retransmits, garbage-collection pauses, page faults, and mechanical vibrations in the rack. So you need to think about how to summarize that distribution.

The arithmetic mean is the usual summary and the wrong one: it does not tell you how many users actually experienced a given delay. Percentiles do. Sort the response times; the median (p50) is the halfway point, meaning half of requests were faster and half slower. The p95, p99 and p999 mark the thresholds that 95, 99 and 99.9 percent of requests beat. These high percentiles are the tail latencies, and they matter disproportionately:

• Slow requests are often the requests of your most valuable users. The customer with the most purchases has the most data, so their requests do the most work. Amazon describes response-time requirements for internal services at p999, which affects only 1 in 1,000 requests, precisely because those are the customers who matter most to revenue.
• The book cites Amazon's finding that a 100 ms increase in response time reduces sales by about 1 percent, and a separate report that a one-second slowdown reduces a customer-satisfaction metric by 16 percent.
• Optimizing the p9999 (1 in 10,000) is usually too expensive, because those requests are dominated by random events outside your control.

Percentiles appear in service level objectives (SLOs) and service level agreements (SLAs), contracts that define expected performance and availability. A typical SLA says the service is considered up if the median response time is under 200 ms and the p99 is under 1 s, and requires that to hold at least 99.9 percent of the time. These set expectations for clients and may entitle customers to a refund if violated.

Queueing and head-of-line blocking

A large part of high-percentile response time is queueing delay. A server can process only a few things in parallel (limited by CPU cores, for example), so a small number of slow requests hold up subsequent ones, even when those would be cheap to handle. This is head-of-line blocking: the request stuck at the front of the queue determines how long everything behind it waits, and the client sees a slow response time even though the server-side service time was fast.

Two measurement consequences follow. Measure response time on the client side, because that is where queueing shows up. And when you load-test, the load generator must keep sending requests at the target rate independently of response time. If the test client waits for one response before sending the next, it shortens the queue artificially and hides exactly the behavior you want to measure (a mistake sometimes called coordinated omission).

Tail latency amplification

In a service-oriented architecture, a single user request may fan out to many backend calls, either in parallel or in sequence. The end-user request has to wait for the slowest of the parallel calls. Even if only a small percentage of backend calls are slow, the chance that at least one of them is slow grows with the number of calls, so a higher proportion of end-user requests end up slow. The book calls this tail latency amplification, and it is why a service with a fine p99 can produce a page with a terrible p99.

This is also why p99 of a backend service is a stricter requirement than it looks: if a page makes 100 backend calls, roughly 63 percent of page loads will include at least one call slower than that service's p99.

Monitoring percentiles cheaply

Dashboards should show percentiles over rolling windows (say, the last ten minutes), which means maintaining a running distribution. Naively keeping a sorted list of every response time is expensive. Approximation algorithms such as forward decay, t-digest and HdrHistogram compute good percentile estimates with tiny CPU and memory cost. One trap: you cannot average percentiles from several machines to get an overall percentile. The right approach is to merge the underlying histograms.

Approaches for coping with load

An architecture appropriate for one level of load is unlikely to be appropriate for ten times that load. If you are working on a fast-growing service, you may need to rethink the architecture at every order of magnitude of growth, perhaps more often. The main dichotomy:

	Scaling up (vertical)	Scaling out (horizontal)
What you do	Move to a more powerful machine	Distribute load across many smaller machines, a shared-nothing architecture
Simplicity	High: one box, no distribution	Lower: coordination, replication, partitioning, partial failures
Cost curve	Very large machines are disproportionately expensive	Commodity hardware, roughly linear
Ceiling	Limited by the biggest machine you can buy	Effectively unlimited, but complexity grows
Typical for	Stateful systems (databases) as long as possible	Stateless services early; stateful systems once a single node cannot cope

In practice, good architectures mix the two: a few fairly powerful machines can be simpler and cheaper than many small virtual machines.

Some systems are elastic: they add computing resources automatically when load increases. That is useful when load is highly unpredictable, but manually scaled systems are simpler and have fewer operational surprises. Distributing stateless services across machines is straightforward; distributing stateful data systems introduces a great deal of extra complexity, which is why the conventional wisdom was to keep a database on a single node (scale up) until scaling cost or availability requirements forced you to distribute. The book notes this is changing as tools and abstractions for distributed systems improve, a theme it develops in Parts II and III.

The key point for interviews is that there is no generic, one-size-fits-all scalable architecture. A system handling 100,000 requests per second of 1 KB each looks nothing like one handling 3 requests per minute of 2 GB each, even though the data throughput is the same. The design is driven by the load parameters you expect to dominate: volume of reads, volume of writes, volume of stored data, complexity of data, response-time requirements, access patterns. Scalable architectures are built from general-purpose building blocks arranged in familiar patterns, but which blocks and which pattern depends on those assumptions. Getting the assumptions wrong is wasted effort at best; worse, it is counterproductive, because you optimized for a problem you do not have. In an early-stage startup, the ability to iterate quickly on features almost always matters more than scaling to hypothetical future load.

Maintainability

The majority of the cost of software is not initial development but ongoing maintenance: fixing bugs, keeping systems operational, investigating failures, adapting to new platforms, paying down technical debt, adding features. Many engineers dislike maintaining legacy systems, and legacy systems are unpleasant in their own particular ways. The chapter's aim is to design software so it does not become legacy in that sense. Three design principles help.

Operability

Good operations can often work around the limitations of bad software, but good software cannot run reliably with bad operations. An operations team is responsible for monitoring health and restoring service, tracking down the causes of problems, keeping software and platforms up to date (security patches especially), understanding how systems affect each other so a problematic change can be avoided before it causes damage, anticipating future problems such as capacity exhaustion, establishing good practices and tooling for deployment and configuration management, performing complex maintenance tasks such as migrating between platforms, maintaining security as configuration changes, defining processes that make operations predictable, and preserving organizational knowledge as people join and leave.

A data system can make routine tasks easy by:

• Exposing its runtime behavior and internals with good monitoring.
• Supporting automation and integration with standard tools.
• Avoiding dependency on individual machines, so a machine can be taken down for maintenance while the system keeps running.
• Providing good documentation and an easy-to-understand operational model ("if I do X, Y will happen").
• Offering good default behavior while giving administrators the freedom to override defaults when needed.
• Self-healing where appropriate, but also giving administrators manual control over the system state when needed.
• Exhibiting predictable behavior and minimizing surprises.

Simplicity

As projects grow, they often become very complex and difficult to understand, which slows down everyone who works on them and increases the cost of maintenance. Symptoms of complexity include explosion of the state space, tight coupling of modules, tangled dependencies, inconsistent naming and terminology, hacks aimed at solving performance problems, and special-casing to work around issues elsewhere.

Complexity makes maintenance hard and increases the risk of bugs: when the system is harder to understand and reason about for developers, hidden assumptions, unintended consequences and unexpected interactions are more easily overlooked. The book borrows from Moseley and Marks's paper "Out of the Tar Pit" the distinction between essential complexity (inherent in the problem the software solves) and accidental complexity (arising only from the implementation). Making a system simpler does not necessarily mean reducing its functionality; it means removing accidental complexity.

The best tool for that is abstraction. A good abstraction hides a great deal of implementation detail behind a clean, simple-to-understand facade, and can be reused across many applications, which leads to higher-quality software because improvements to the abstraction benefit every application using it. High-level programming languages hide machine code and CPU registers; SQL hides on-disk and in-memory data structures, concurrent requests from other clients, and inconsistencies after crashes. Finding good abstractions for distributed systems is hard, which is why the book spends so much time on it.

Evolvability

Your system's requirements will change. You learn new facts, previously unanticipated use cases emerge, business priorities shift, users request new features, new platforms replace old ones, legal or regulatory requirements change, growth forces architectural changes. Evolvability (also called extensibility, modifiability or plasticity) is the ease with which you can adapt the system to those changes.

At the level of code organization, Agile practices such as test-driven development and refactoring provide a framework for adapting to change. The book's interest is in the same property at the level of a larger data system, where the mechanisms are different: schema evolution (Chapter 4), being able to migrate data between systems, and architectures that let you swap or add components. Simplicity and good abstractions help here too: simple and easy-to-understand systems are usually easier to modify than complex ones.

Design trade-offs

The three properties pull against each other and against cost, and knowing which lever you are pulling is most of the job.

Decision	Favors	Costs	When to choose it
Hardware redundancy only (RAID, dual PSUs)	Reliability with minimal software complexity	Whole-machine failures still cause downtime; does not scale to thousands of nodes	Small deployments on a few machines
Software fault tolerance (replication, failover)	Reliability and rolling upgrades	Replication lag, consensus, more complex operations	Any system where downtime is expensive or that spans many machines
Fan-out on write (precomputed timelines, caches)	Read latency at high read/write ratios	Heavy writes for high-fan-out items; stale caches	Read-heavy workloads with bounded fan-out; use a hybrid for the heavy tail
Join on read	Simplicity, fresh data, cheap writes	Read cost grows with fan-out	Low read volume or when data must be fresh
Scaling up	Simplicity, strong single-node guarantees	Price grows faster than capacity; hard ceiling	Stateful stores until a single node genuinely cannot cope
Scaling out	Elastic capacity, fault tolerance	Partitioning, replication, partial failures	Stateless tiers early; stateful tiers when forced
Elastic auto-scaling	Unpredictable load	Operational surprises, cost spikes, slow warm-up	Spiky, unpredictable traffic
Optimizing p999	Highest-value users	Diminishing returns against random noise	When the SLO or the business explicitly demands it

A useful habit in a design discussion: state the load parameters first, name the percentile you are designing for, and only then pick components.

Interview angle

• What is the difference between a fault and a failure, and why does it matter? A fault is a component deviating from spec; a failure is the system no longer delivering its service. The distinction frames the design goal: you cannot prevent faults (disks die, humans mistype), so you design mechanisms that stop a fault from becoming a failure, and you test them by deliberately inducing faults.
• Why are software faults more dangerous than hardware faults in a replicated system? Hardware faults are random and roughly independent, so replication across machines handles them. A software bug is present on every replica, so a single trigger (a bad input, a leap second, a dependency going slow) takes out all replicas at the same moment. Replication does not protect against correlated faults; diversity, isolation, careful rollout and invariant checking do.
• Why not report average latency? The mean does not tell you how many users saw a given delay and is skewed by outliers in both directions. Percentiles answer the user-facing question directly: p99 of 1 s means 1 in 100 requests is slower than a second. High percentiles also reflect your heaviest, often most valuable, users.
• A backend has a p99 of 50 ms, but the page that calls it has a p99 of 2 s. What is happening? Tail latency amplification: the page issues many backend calls and waits for the slowest. With 100 parallel calls, most pages include at least one call above the backend p99. Fixes include reducing fan-out, hedged or speculative requests, tighter per-call timeouts with fallbacks, and designing SLOs for the backend at a higher percentile than the page's.
• Design the home timeline for a Twitter-like service. Identify the load parameters: roughly 5k to 12k tweets per second, 300k timeline reads per second, average fan-out about 75, a heavy tail of users with tens of millions of followers. Fan-out on write into per-user timeline caches makes reads cheap at about 345k cache writes per second. Exclude celebrity accounts from fan-out and merge their recent tweets at read time, so a single tweet never triggers 30 million writes.
• What is head-of-line blocking and how does it affect your measurements? A server handles a limited number of requests concurrently, so a few slow requests delay cheap requests queued behind them. The client sees high response times even though server-side service time was low. Therefore measure on the client, and load-test with an open-loop generator that keeps sending at the target rate regardless of responses.
• When would you scale up rather than out? For stateful systems, as long as a single node can handle the load with acceptable availability. A single node avoids replication lag, partitioning and distributed-transaction complexity. You scale out when the biggest affordable machine is no longer enough, when you need fault tolerance across machines, or when you want rolling upgrades without downtime.
• How would you reduce the chance that a human error causes an outage? Make the dangerous path hard (good abstractions, confirmations, least privilege), make the safe path easy (sandbox with real data, staged rollouts, feature flags), make recovery fast (one-command rollback, tools to recompute derived data), and make problems visible early (telemetry, alerts on error rates and invariants).
• What does "maintainable" mean concretely? Operable: monitoring, automation, no dependency on single machines, good defaults, predictable behavior. Simple: accidental complexity removed through good abstractions. Evolvable: easy to change requirements, data formats and components without a rewrite.

Pitfalls and misconceptions

• "Our system is scalable." Scalability is not a property a system has or lacks; it is a set of answers to "what happens if load parameter X grows tenfold?" Name the parameter.
• Treating the average as representative. A mean response time of 100 ms is compatible with 1 percent of users waiting 10 s. Percentiles are the only honest summary; and you cannot average percentiles across machines, you have to merge histograms.
• Assuming replication protects against all faults. Replication handles independent hardware faults. Correlated software faults (same bug on every node) and operator errors (a bad config pushed everywhere) go straight through it.
• Load-testing with a closed loop. If your load generator waits for each response before sending the next request, slow responses throttle the generator, the queue never builds up, and your measured tail latency is optimistic.
• Optimizing for the average user in a fan-out system. Twitter's average fan-out of 75 is fine; the celebrity with 30 million followers is the design problem. Always ask about the distribution, not the mean.
• Scaling out prematurely. Distributing a stateful system introduces replication lag, partitioning and consensus problems. Until a single node is actually the bottleneck, that complexity is pure cost.
• Believing elasticity is free. Auto-scaling helps with unpredictable load but adds operational surprises (cold caches, cost spikes, scale-up lag). Manual scaling is simpler when load is predictable.
• Equating simplicity with fewer features. The target is accidental complexity, not functionality. A good abstraction can make a feature-rich system simpler than a stripped-down one with tangled internals.

Key terms

Term	Meaning
Data-intensive application	An application whose main challenges are data volume, complexity or rate of change rather than CPU
Fault	One component deviating from its specification
Failure	The system as a whole failing to provide its service to the user
Fault tolerance / resilience	Designing so that faults do not cause failures
Rolling upgrade	Updating nodes one at a time so the system stays available throughout
Cascading failure	A fault in one component triggering faults in others, spreading through the system
Telemetry	Detailed monitoring of performance metrics and error rates used for early warning and diagnosis
Load parameter	A number describing load (requests/sec, read/write ratio, fan-out, active users)
Fan-out	The number of downstream writes or calls one incoming request triggers
Throughput	Records processed per unit time; the key metric for batch systems
Response time	Time between a client sending a request and receiving the response, including network and queueing delay
Latency	Strictly, the time a request waits to be handled; often used loosely as a synonym for response time
Percentile (p50, p99, p999)	The threshold a given fraction of requests complete within; p50 is the median
Tail latency	Response time at high percentiles (p99 and above)
Tail latency amplification	A request that fans out to many backends is as slow as the slowest backend call
Head-of-line blocking	A few slow requests delaying cheaper requests queued behind them
SLO / SLA	Service level objective (internal target) / agreement (contractual) on performance and availability
Scaling up / scaling out	Moving to a more powerful machine / distributing across many machines (shared-nothing)
Elastic	Automatically adding resources when load increases
Operability, simplicity, evolvability	The three components of maintainability
Accidental complexity	Complexity arising from the implementation rather than the problem itself

Further reading

• DDIA, Chapter 1 (and Kleppmann's chapter reference list at https://github.com/ept/ddia-references).
• Jeffrey Dean and Luiz André Barroso, "The Tail at Scale", Communications of the ACM, 2013. The canonical treatment of tail latency amplification and hedged requests.
• Ben Moseley and Peter Marks, "Out of the Tar Pit", 2006. Source of the essential versus accidental complexity distinction used in the Simplicity section.
• Gil Tene, "How NOT to Measure Latency" (talk). The clearest explanation of coordinated omission and why percentiles from a closed-loop load generator lie.
• Betsy Beyer et al. (eds.), Site Reliability Engineering: How Google Runs Production Systems, O'Reilly, 2016 (free online at https://sre.google/sre-book/table-of-contents/). The chapters on SLOs, monitoring and cascading failures expand the Reliability and Operability sections.
• James Hamilton, "On Designing and Deploying Internet-Scale Services", USENIX LISA 2007. Practical operability advice that aligns closely with this chapter.
• Raffi Krikorian, "Timelines at Scale" (QCon talk, 2012). The source of the Twitter fan-out numbers.
• David Oppenheimer, Archana Ganapathi and David Patterson, "Why Do Internet Services Fail, and What Can Be Done About It?", USENIX USITS 2003. The study behind the claim that operator error is the leading cause of outages.