Appendices
The five appendices condensed: the availability table, a collection of SRE best practices for production services, an example incident state document, an example postmortem, a launch coordination checklist, and example production meeting minutes.
The appendices are the book's reference material: the tables and templates that the chapters refer to. They are short, practical and worth having in one place, so they are summarised here as the book's final section.
Appendix A: Availability table
Availability is usually quoted in nines. The table converts a target into the downtime it permits.
| Availability | Per year | Per quarter | Per month | Per week | Per day |
|---|---|---|---|---|---|
| 90% | 36.5 days | 9 days | 3 days | 16.8 hours | 2.4 hours |
| 99% | 3.65 days | 21.6 hours | 7.2 hours | 1.68 hours | 14.4 minutes |
| 99.9% | 8.76 hours | 2.16 hours | 43.2 minutes | 10.1 minutes | 1.44 minutes |
| 99.95% | 4.38 hours | 1.08 hours | 21.6 minutes | 5.04 minutes | 43.2 seconds |
| 99.99% | 52.6 minutes | 12.96 minutes | 4.32 minutes | 60.5 seconds | 8.64 seconds |
| 99.999% | 5.26 minutes | 1.30 minutes | 25.9 seconds | 6.05 seconds | 0.87 seconds |
Two points follow. A five-minute human response time cannot deliver 99.99 percent on its own, which is why automation and redundancy carry the higher nines. And each extra nine is roughly ten times harder; chapter 3 asks whether users can tell the difference before you pay for it.
Appendix B: A collection of best practices for production services
A checklist of what Google SRE considers table stakes, grouped by theme.
- Fail sanely. Sanitise and validate configuration before it is loaded, and fall back to the previous known-good configuration if the new one is bad. Prefer serving stale or degraded data to failing completely.
- Progressive rollouts. Roll out in stages (canary, then a fraction, then everything), with monitoring at each step and an automatic stop on regressions. Never push a change globally in one step.
- Define SLOs like a user. Measure from the user's point of view, use percentiles, and pick the targets users actually need.
- Error budgets. Agree them in advance and use them to govern release velocity, so disagreements are settled by the number rather than by escalation.
- Monitoring. Alert on symptoms that need a human now; use tickets for the rest; keep logs for debugging. Keep the rules simple and prune the ones that do not fire or are not acted on.
- Postmortems. Blameless, written for every significant incident, with action items that are tracked to completion.
- Capacity planning. Provision for N+2 so you can lose one replica to maintenance and one to failure. Validate forecasts against actual demand and load-test regularly, because real capacity drifts.
- Overloads and failure. Load shed by criticality, use deadlines and propagate them, bound queues and retries, and test the overload behaviour before you need it.
- SRE teams. Keep operational load under 50 percent, keep on-call to the limits in chapter 11, and give pages back to developers when the service is too noisy.
Appendix C: Example incident state document
The template from chapter 14, filled in for a fictional outage. Sections: incident commander and other roles with names, a one-line summary of the current state, a timestamped log of actions and findings, the current theory of the cause, what is being done now, exit criteria for the incident, and a list of communications sent. The document is live during the incident and becomes the skeleton of the postmortem.
Appendix D: Example postmortem
A complete postmortem for the Shakespeare service, following the template from chapter 15. Sections: title and authors, status, summary, impact (duration, users affected, revenue), root causes, trigger, resolution, detection, action items with owners, types and priorities, lessons learned split into what went well, what went wrong and where we got lucky, and a timeline. The example shows a cascading failure triggered by an unexpected traffic spike (a new sonnet discovered, searches surged), with a post-incident focus on capacity planning and load shedding.
Appendix E: Launch coordination checklist
The checklist from chapter 27, in full. Headings: architecture (request flow, dependencies, infrastructure), integration (DNS, load balancing, new endpoints), capacity planning (estimated traffic, load testing, growth), failure modes (single points of failure, dependency failures, degraded modes), client behaviour (retries, abusive clients, backoff), processes and automation (on-call, runbooks, rollback), development process (code review, CI, config in version control), external dependencies (vendors, partners, legal), and rollout planning (stages, metrics, go or no-go criteria).
Appendix F: Example production meeting minutes
Minutes from a production meeting, following the agenda in chapter 31: who attended, upcoming production changes, metrics against SLO for the week, outages with links to postmortems, paging events with a note on whether each was actionable, non-paging events, and a review of prior action items with their status. The example is short, which is the point: the meeting is a weekly check that the team knows the state of its service and has owners for what needs to change.