Site reliability engineering

Also known as: SRE

On this page 7

Site reliability engineering (SRE) is a discipline that applies software engineering methods to operations, treating the reliability of production services as a measurable, engineered property managed through service level objectives, error budgets, automation and systematic learning from incidents. In DevOps, SRE is often described as a concrete implementation of DevOps principles: it gives shared ownership of production a precise vocabulary, numeric targets and a decision rule for balancing speed against stability.

The practice originated at Google in the early 2000s, when a team of software engineers was asked to run production systems. Its central insight is that 100 percent reliability is the wrong target: users cannot tell the difference between very high and perfect availability, while chasing perfection slows every change. SRE instead defines how reliable is reliable enough and spends the remaining margin on innovation.

SLIs, SLOs and error budgets

SRE starts by measuring what users actually experience. A service level indicator (SLI) is a ratio of good events to valid events, such as the share of HTTP requests answered successfully within 300 milliseconds. A service level objective (SLO) sets a target for that indicator over a window, typically 28 or 30 days. An SLA is a contractual promise with penalties and is usually looser than the internal SLO.

The error budget is what the SLO leaves over: with a 99.9 percent objective, 0.1 percent of requests may fail. As long as budget remains, teams ship freely; when it is exhausted, they shift effort from features to reliability work. The table translates common objectives into allowed downtime.

SLOAllowed unavailabilityTypical fit
99%About 7.2 hoursInternal tools, batch systems
99.5%About 3.6 hoursNon-critical customer features
99.9%About 43 minutesMost customer-facing services
99.95%About 22 minutesImportant commercial services
99.99%About 4.3 minutesCore platforms, payments, identity
Availability objectives and their error budgets over 30 days

Each additional nine costs disproportionately more in redundancy, automation and on-call effort, so objectives should follow business need rather than ambition.

Core practices of SRE teams

Beyond SLOs, SRE rests on a small set of recurring practices that together turn operations into engineering work. The table summarizes them.

PracticeWhat it involvesIntended outcome
Toil reductionAutomating manual, repetitive, non-lasting workToil kept below about 50 percent of time
Monitoring and alertingAlerts on symptoms and SLO burn rate, not causesPages that always require action
Incident responseDefined roles such as incident commanderFaster, calmer restoration
Blameless postmortemsWritten analysis of contributing factorsSystemic fixes, not individual blame
Capacity planningForecasting demand and load testingNo outages from predictable growth
Release engineeringProgressive rollouts, automatic rollbackSmaller blast radius per change
Principal SRE practices

Monitoring commonly follows the four golden signals: latency, traffic, errors and saturation. Alerting on error budget burn rate, for example paging when the budget would be gone in hours rather than weeks, keeps alerts tied to real user impact and reduces noise.

Burn-rate alerting expresses speed of failure relative to the objective. When failures arrive at exactly the pace the SLO tolerates, the budget runs out on the last day of the window; when they arrive fourteen times faster, an hour of trouble already costs about one fiftieth of the monthly allowance. SRE teams usually pair a short evaluation window that pages with a longer one that only files a ticket, separating emergencies from gradual decline.

SRE overlaps with several neighboring approaches, and the boundaries are organizational as much as technical. The comparison shows the main differences in focus.

ApproachPrimary focusKey measures
Site reliability engineeringReliability of running servicesSLOs, error budgets, toil share
DevOpsCulture of shared delivery and operationsDORA metrics such as deployment frequency
Platform engineeringInternal developer platforms and self-serviceDeveloper productivity, adoption
Traditional operationsKeeping systems running through manual proceduresUptime, ticket volume
SRE and neighboring disciplines

DevOps describes what culture to build; SRE prescribes specific mechanisms for one part of it. The two are complementary, and many organizations use DORA metrics for delivery performance alongside SLOs for production reliability.

Organizational models for SRE

There is no single team structure for SRE. The right model depends on company size, the number of services and how mature development teams are in running their own systems.

ModelHow it worksMain risk
Embedded SREReliability engineers sit inside product teamsInconsistent practices between teams
Centralized SRE teamOne team supports many servicesBecomes a bottleneck or a renamed ops team
Consulting SREExperts coach teams that own their servicesAdvice without operational authority
Product team SRE practicesDevelopers adopt SLOs and on-call themselvesReliability work loses to feature pressure
Common ways to organize SRE

A sound arrangement includes an explicit error budget policy agreed with product management, which defines what happens when a service exhausts its budget, such as a temporary feature freeze.

Common mistakes when adopting SRE

  • Renaming the operations team: a new title without engineering time, automation goals and SLOs changes nothing.
  • Too many SLOs: dozens of objectives dilute attention; a few user-centered indicators per service work better.
  • Measuring from the server, not the user: internal health checks can be green while users see errors.
  • Ignoring the error budget policy: if exhausted budgets never slow releases, the SLO becomes decoration.
  • Alerting on every cause: CPU or disk alerts without user impact create fatigue and missed real incidents.
  • Blame after incidents: punishing individuals hides information and repeats the same failures.

Successful adoption is usually incremental: define one meaningful SLO for a critical service, measure it honestly, agree on the policy, and expand from there.

A plain-language summary

Imagine an airline that knows a tiny share of flights will always be delayed. Instead of promising perfection, it sets a clear goal, such as 99 percent on time, and measures every flight against it. If the goal is met, new routes can be added. If it is missed, the airline fixes punctuality first.

Site reliability engineering runs online services the same way. Engineers agree on how reliable a service must be, track it constantly, and use automation and careful review of failures to keep it there.

An example from practice

An online banking provider defines an SLO for its login service: 99.95 percent of login requests succeed within one second over 30 days, which leaves about 22 minutes of equivalent failure. Alerts fire when the error budget is burning fast enough to be exhausted within a few hours.

Midway through the month, two faulty releases consume 80 percent of the budget. Following the agreed policy, the team pauses feature releases for the login service, adds automated canary analysis to the deployment pipeline and fixes a database connection leak found in the blameless postmortem. In the following months the service stays within budget, and release frequency actually increases because problems are caught earlier.