Guides & resources

Practical knowledge for the humans building more reliable systems.

Report

How to achieve great incident management

What does reliable software look like in the age of AI? We analyzed 350,000 recent incidents to understand how top-performing teams handle incidents.

Top-performing teams page fewer than 4 times per incident, at every company size.

1.68.23.424.53.960.5

Pages per declared incident, per team. Company size measured by annual revenue.

Guide

Incident response

Incident response works best as a practiced system: clear roles, shared priorities, reliable communication, and a process that improves after every incident.

Start with the overview

Lifecycle process

A dependable response process gives SRE teams a shared path from the first alert through recovery and review, even when the incident itself is unfamiliar.

Team roles

Clear roles keep responders from colliding under pressure. Give technical work, communication, documentation and leadership an owner, with matching workflows.

Runbooks

Runbooks turn known failure modes into executable steps, but they only earn trust when they are easy to find, safe to follow, and kept current.

Metrics

How to measure MTTR accurately: formulas for MTTD, MTTA, MTTM and MTTR, a worked mean vs median example, and the pitfalls that skew incident metrics.

14 chapters in this guide

Guide

AI SRE and automated RCA

Learn how AI SRE agents investigate incidents, analyze likely root causes, and support remediation. Explore capabilities, limitations, and adoption steps.

Start with the overview

Architecture

An AI agent can have the right code and the wrong context. See how AI SRE architecture connects evidence, policy, execution, verification, and rollback.

Implementation guide

A successful AI SRE rollout builds trust in stages, beginning with better incident context and introducing controlled action only when the workflow is ready.

AI SRE vs AIOps

AI SRE vs AIOps compared on eight axes: what each does, when it acts and how it is measured, plus how they work together and which to start with.

Metrics & ROI

The value of AI SRE appears first in faster context, safer actions, and less on-call toil, not necessarily in a dramatic change to MTTR.

12 chapters in this guide

Put it into practice

All webinars

Guide

Blameless retrospectives

A good postmortem turns one incident into better systems, clearer ownership, and fewer repeat failures by helping the team understand what really happened.

Start with the overview

Timelines

A reliable timeline replaces conflicting memories with a shared sequence of events, making delays, decisions, and missed signals easier to understand.

Root cause analysis

How engineering teams run root cause analysis on production incidents: methods that work, turning contributing factors into owned fixes, and preventing repeats.

Running a postmortem meeting

How to run a post-incident review meeting: roles, a 60-minute agenda, blameless facilitation questions and how to make sure action items get finished.

Action items

Why post-incident action items stall after the retrospective, and the five rules, review cadence and follow-through metrics that get them finished and verified.

8 chapters in this guide

Guide

On-call operations

What to look for in on-call software: who owns an alert, who gets paged, and what happens when nobody answers the first page.

Start with the overview

Schedules and rotations

A well-designed rotation makes coverage predictable and recovery possible; a poor one turns reliability into an uneven tax on the same few engineers.

On-call pay

On-call compensation works when it reflects the real burden of readiness and gives teams a transparent way to account for time, disruption, and workload.

Alert fatigue

Alert fatigue begins when every signal demands attention, leaving responders slower to recognize the few notifications that truly require action.

Escalation policies

Escalation should remove uncertainty during an incident by making it clear who responds first, when backup joins, and how coverage is maintained.

11 chapters in this guide

All resources

Resource type

Your privacy choices

Rootly uses cookies from advertising partners to measure our ads and to show ads for Rootly on other sites. Some US state laws call that selling or sharing personal information. You can opt out of it for this browser here.

Analytics we use to improve this site stay on. Read Rootly's Privacy policy for more information.