# How to evaluate AI features in incident management

URL: https://rootly.com/ai-sre-guide/evaluate-ai-incident-management
Pillar: AI SRE
Bundle: https://rootly.com/llms/ai-sre.txt

> A weighted scorecard, 12 vendor questions, red flags and a two-week proof of concept for judging AI features in incident management software.

Evaluate AI features in incident management software on your own incidents. Check whether the root cause is right and shows its evidence, which actions wait for a human, what data the AI reads and keeps, and what it costs. Then run a two-week proof of concept on real alerts. Rootly AI SRE is the worked example throughout.

Demos are a poor test of AI in incident management. Every vendor shows a clean incident with a clear cause. Your on-call engineers deal with partial telemetry, stale runbooks and causes that only show up in a deploy log. The scorecard, questions and trial plan below test for that.

## Key takeaways

- Weight root-cause accuracy and evidence highest. A fast wrong answer costs more than no answer.
- Ask what the AI can change, and with whose permissions, before you ask what it can read.
- An honest "inconclusive" is a feature. Penalize tools that always name a cause.
- Test on resolved incidents whose cause you already know, then on live alerts.
- Rootly AI SRE reports a root cause with a High, Medium or Low confidence tier and cites the evidence, or returns Inconclusive or Blocked and says what is missing.

## Scorecard for AI features in incident management

Score each vendor from 1 to 5 on each row, multiply by the weight, and add the results. A perfect score is 500. Move weight toward the rows that hurt most in your last five incidents, and keep the total at 100%.

| Criterion | Weight | What good looks like | Question to ask |
| --- | --- | --- | --- |
| **Root-cause accuracy and evidence** | 25% | Names a cause only when the evidence supports it, shows a confidence level, cites each source, and lists what it ruled out | On five of our resolved incidents, how often does it reach our answer, and what does it show when it can't? |
| **Actions with human approval** | 15% | Reads freely within scope, and waits for a person before anything destructive or customer-facing | Which actions run without a person, which need a click to confirm, and which can it never take? |
| **Data access and privacy** | 15% | Clear answers on model providers, training use, PII handling, retention and processing region | Is our data used to train models? How long are prompts and results kept, and where does inference run? |
| **Context sources** | 15% | Reads the tools responders open during an incident: observability, code and deploys, feature flags, tickets, runbooks and past incidents | Which of our tools can it query during an investigation, and what can't it see? |
| **Integration depth** | 10% | Works in Slack or Microsoft Teams and in the incident record, with results where responders already look | Can a responder start an investigation, read the result and ask a follow-up without leaving the incident channel? |
| **Auditability** | 10% | Every AI action is logged and attributed to the person who asked, and every run is listed with its trigger and outcome | Who is recorded as the actor when the AI changes something? Can we review every automatic run? |
| **Latency** | 5% | A useful first hypothesis before the responder would have found one by hand | How long does a typical investigation take on our data, and does it stream progress while it works? |
| **Cost model** | 5% | Pricing you can forecast at your incident volume, with usage limits stated up front | Is AI priced per seat, per investigation or per token? What happens when we hit a limit? |

For the incident platform as a whole (alert routing, workflows, status pages and retrospectives), use the [incident management tool buying scorecard](https://rootly.com/incident-response/tools#incident-management-tool-buying-scorecard). This page goes deeper on the AI row.

## 12 questions to ask AI incident management vendors

1. Can you run your AI on five of our resolved incidents, using only the data available at the time, and show us where it agreed and disagreed with our retrospectives?
2. What does a result look like when the evidence doesn't confirm a cause?
3. How is the confidence level decided, and can the model declare a root cause on its own?
4. Which of our tools can it query during an investigation, and which sources can't it see at all?
5. What can it change in our systems, and under whose credentials does it act?
6. Which actions need a person to confirm, and can we keep automatic runs off until we trust them?
7. Which models do you use, through what gateway, and is our data used to train or fine-tune them?
8. How long do you keep prompts, results and conversation history, and can we set our own retention?
9. Can we use our own LLM API keys?
10. How are AI actions recorded in the audit trail, and can we export them?
11. How do responders tell you a result was wrong, and what changes when they do?
12. How is AI priced, and what usage limits apply?

## Red flags when evaluating AI incident management

- **It always names a root cause.** Real incidents often end with partial evidence. A tool that never says "inconclusive" is guessing some of the time.
- **No evidence behind the claim.** If you can't click from a conclusion to the log line, metric, deploy or commit that supports it, you can't check it at 3 a.m.
- **"Autonomous remediation" with no stated boundary.** Ask exactly what it runs, with which credentials, and who approved it.
- **Vague answers on data.** "Enterprise-grade security" isn't an answer to which model provider sees your data and how long it is kept.
- **The demo runs only on vendor data.** If they won't replay your incidents, the trial isn't measuring your environment.
- **A separate console.** If responders must leave the incident channel to use the AI, they will stop using it.
- **No way to rate results.** Without feedback on each run, nobody can tell whether accuracy is improving.
- **Pricing that depends on incident volume you can't predict**, with no stated limits.

## Two-week proof-of-concept plan

Set your success metrics before day one, and score every vendor on the same incidents.

| Days | What to do | Exit check |
| --- | --- | --- |
| 1–2 | Connect a code or deploy source and your main observability provider with read-only identities. Write down team context the AI can't learn from data: service aliases, the monitor that measures customer impact, where deploys are recorded | Each connector authenticates and returns data for a test query |
| 3–5 | Replay 5–10 recent, resolved alerts whose cause your team already knows. Include one where the right answer was "not enough evidence" | Results recorded against your known answers |
| 6–7 | Review the misses with the engineers who ran those incidents. Connect the missing source or fix the context, then rerun | Each miss has a reason: missing source, wrong context or wrong reasoning |
| 8–12 | Run on live alerts for one alert class, with a person starting each investigation | Responders rate every result |
| 13–14 | Score the vendor on the scorecard with the on-call engineers who used it, and decide whether to allow automatic runs for that alert class | Scorecard complete, decision recorded |

Track these success metrics:

- **Agreement with known causes:** the share of replayed incidents where the AI reached your team's answer.
- **Honest misses:** the share of misses where it reported inconclusive instead of a wrong cause.
- **Evidence quality:** the share of conclusions where the cited evidence says what the report claims.
- **Time to a useful hypothesis**, compared with how long responders took on the same incidents.
- **Responder rating:** the average score responders gave live results.
- **Unapproved changes:** zero writes to production or customer channels that nobody confirmed.

Use [AI SRE metrics and ROI](https://rootly.com/ai-sre-guide/metrics-and-roi) to carry these into a business case, and the [AI SRE implementation guide](https://rootly.com/ai-sre-guide/implementation-guide) for the rollout after the trial.

## What AI should and shouldn't do in an incident today

Judge any AI in incident management by three lists: what it does on its own, what it does only after a person approves, and what it shouldn't do yet. The table below sets out a sensible default. The same three-column view, for Rootly specifically and with every row sourced, is in [what Rootly AI SRE does today, with approval and not at all](https://rootly.com/ai-sre-guide/ai-sre-vs-aiops#what-rootly-ai-sre-does-today-with-approval-and-not-at-all).

| Let AI do it today | Do it with human approval | Don't hand it to AI yet |
| --- | --- | --- |
| Gather context from alerts, logs, metrics, deploys and past incidents | Start investigations automatically for an alert class, once manual results hold up | Declare a root cause the evidence doesn't support |
| Test competing causes and report the most likely one with its evidence | Publish a status page update or message customers | Roll back deploys, restart services or change infrastructure on its own |
| Draft incident summaries, catchups and stakeholder updates | Delete records or change incident severity | Write and merge code fixes without review |
| Draft retrospective sections from the timeline | Hand work to a coding agent under that agent's own permissions | Decide reliability trade-offs an SRE is accountable for, such as SLOs |
| Find similar past incidents | Recall notes about your systems that your team has reviewed | Treat a chat thread or a memory note as proof |

In Rootly, the first column is what Rootly AI SRE does on its own. Destructive actions in the incident channel wait for a responder to click Confirm, and Rootly AI's native tools don't run kubectl, roll back a deploy, restart a pod, flip a feature flag or write code ([what Rootly AI can't do](https://docs.rootly.com/ai/rootly-in-slack/what-to-use-rootly-for)).

## How teams evaluate Rootly AI SRE

Rootly's own setup guide reads like a proof-of-concept plan, so teams can run the trial above inside the product. Each point links to the Rootly documentation that describes it.

- **Enablement.** Your Rootly account team enables AI SRE for each Rootly team. There is no self-service switch ([getting started](https://docs.rootly.com/ai/ai-sre/getting-started)).
- **Calibrate on a known alert.** Rootly recommends connecting a code or deploy source, such as the [GitHub connector](https://docs.rootly.com/ai/connectors/github), and your main observability provider first, then starting an investigation on a recent resolved alert whose cause your team knows ([getting started](https://docs.rootly.com/ai/ai-sre/getting-started)).
- **Manual first, then Auto-run.** Each [investigation rule](https://docs.rootly.com/ai/ai-sre/investigation-rules) runs in Manual, Auto-run or Paused mode. Manual rules never start a run by themselves, so a person starts each investigation until your team promotes the rule.
- **Evidence you can check.** Rootly AI SRE tests competing causes and challenges the leading one ([hypothesis testing](https://docs.rootly.com/ai/ai-sre/hypothesis-testing)). The final outcome depends on checks Rootly runs against the recorded evidence, so the model can't declare a root cause on its own ([overview](https://docs.rootly.com/ai/ai-sre/overview)). Each evidence item is marked Decisive or Supporting, with a View link to the source tool ([reading the result](https://docs.rootly.com/ai/ai-sre/reading-the-result)).
- **Honest outcomes.** A result is Root cause identified with a High, Medium or Low confidence tier, or Inconclusive, or Blocked with recommended next steps that name the source to connect or re-authorize.
- **Context sources.** [AI connectors](https://docs.rootly.com/ai/connectors/overview) cover observability (such as Datadog, Grafana, New Relic, Honeycomb and Sentry), code and CI, cloud, feature flags (LaunchDarkly), work tracking, knowledge tools and any public MCP server. A live investigation doesn't search Slack channel history or read third-party status pages ([evidence sources](https://docs.rootly.com/ai/ai-sre/evidence-sources)).
- **Rate every result.** Responders score each investigation from 0 to 5 under "How accurate was this investigation?", and the Investigations page lists every run with its outcome, confidence, feedback, trigger and duration ([reading the result](https://docs.rootly.com/ai/ai-sre/reading-the-result)).
- **Data and models.** Rootly AI runs on frontier models from Anthropic and OpenAI through a Rootly-managed gateway. PII is scrubbed from outgoing prompts, inference runs in US-based infrastructure, and customer data isn't used to fine-tune base models. AI session messages are removed after 90 days without activity ([data privacy](https://docs.rootly.com/ai/data-privacy-for-rootly-ai)).
- **No bring-your-own LLM keys yet.** Rootly doesn't accept your own LLM API keys today. The data privacy documentation lists BYOK support as on the roadmap.
- **Audit trail.** Rootly-native actions a person starts through Rootly AI are recorded in the standard audit trail under that person's name, and Rootly AI is in scope of Rootly's SOC 2 Type II controls. Actions through an external connector appear in the provider's own audit history under the connected account ([data privacy](https://docs.rootly.com/ai/data-privacy-for-rootly-ai)).
- **Write access.** Rootly's documentation says AI SRE isn't guaranteed to be read-only: a connector tool can change data when the provider account allows it, and automatic runs have no person in the loop. Connect with read-only identities and expose only tools that are safe to run unattended.
- **Cost.** AI SRE requires Rootly Incident Response, is priced on request and is included in Enterprise ([pricing](https://rootly.com/pricing)). Each tenant has a per-minute token budget enforced server-side.

For a broader framework that also covers AIOps platforms, see [how to evaluate AIOps and agentic AI tools](https://rootly.com/blog/a-guide-to-evaluating-aiops-and-agentic-ai-tools).

## Frequently asked questions

### How should we evaluate AI features in incident management software?

Score each vendor on root-cause accuracy and evidence, actions that wait for human approval, data access and privacy, context sources, integration depth, auditability, latency and cost. Then run a two-week proof of concept on your own resolved and live alerts. Rootly AI SRE is built for that test: it cites the evidence for each conclusion and returns Inconclusive when the evidence doesn't confirm a cause.

### What is the most important AI feature in incident management?

Root-cause accuracy with visible evidence. A responder has to check an AI conclusion before acting on it, so a result without sources slows the incident down. Look for a confidence level, cited evidence and a list of what was ruled out, as in Rootly AI SRE's results.

### Should AI take remediation actions during an incident?

Not on its own, today. Let it gather context and recommend the next step, and keep a person on anything destructive or customer-facing. In Rootly, destructive actions in the incident channel wait for a responder to click Confirm, and Rootly AI's native tools don't roll back deploys or restart services.

### How long should a proof of concept for AI incident management take?

Two weeks is enough for one alert class: a few days to connect sources, a few days replaying resolved incidents with known causes, and a week on live alerts. Rootly's AI SRE setup follows the same path, from a Manual rule on a known alert to Auto-run once results hold up.

### Can we use our own LLM API keys with Rootly AI SRE?

No. Rootly AI SRE runs model inference through a Rootly-managed LLM gateway with Rootly's model credentials. Rootly's data privacy documentation lists bring-your-own-key support as on the roadmap.

### Is our incident data used to train AI models?

Ask every vendor this in writing. For Rootly, customer data is processed in context for each request and isn't used to fine-tune base models, according to Rootly's data privacy documentation.