In focus
“Human in the Loop” is the new “Pay Attention” for AI Ops
Orgs that build AI policy around a “human-in-the-loop” accountability model will find mixed results, and little leverage when they try to improve reliability.
Perspectives on AI, engineering, and the humans keeping systems running.
In focus
Orgs that build AI policy around a “human-in-the-loop” accountability model will find mixed results, and little leverage when they try to improve reliability.
| Phase | Owner |
|---|---|
| Capture the timeline | Automated, during the incident |
| Draft the narrative | Automated, clearly labelled a draft |
| Decide the contributing factors | Human, in the review |
| Choose which to fix | Human, with the people who own the systems |
What automation reliably contributes to a post-incident review, what it cannot, and how to run retrospectives that produce fixes instead of documents.
“Autonomy is not one switch; it is a dial per action.”
With reliability platforms as code and agents that rank probable causes from telemetry, catalog and incident history, could 2027 end human-in-the-loop fixes?
| Ask this | Because |
|---|---|
| “Show me a case where it was wrong” | A vendor who cannot produce one has not looked, or is not telling you |
| “What data does it read, and what is it blind to?” | The boundary of the input is the boundary of the capability |
| “What happens when it has no confident answer?” | Abstaining is a feature. Guessing is not |
What AI SRE agents reliably do today, what they still get wrong, and how to evaluate a vendor claim without taking the demo at face value.

We built Doom Agent Arena, an open-source benchmark where AI agents battle in Doom via MCP. Here's what it taught us about AI-assisted incident response.
Brent Chapman argues that AI-written post-incident reviews quietly automate away the learning. We build the AI that drafts them.
Results from running Claude Sonnet 4.6 through SRE-skills-bench on release day, plus what its adaptive thinking means for agentic incident investigation.
Gemini 3 Pro scored about 4% more accurate than competing models on SRE-skills-bench, ending OpenAI's long lead on SRE tasks. See the findings and methodology.
What KubeCon North America 2025 in Atlanta showed about AI taking operational actions, and the reliability lessons it holds for on-call teams, from Rootly.
Why machine learning systems fail quietly with no alerts, drawing on an MLOps conversation with Maria Vechtomova about monitoring gaps and reliability for ML.
The new edition of our benchmark features Terraform tasks across AWS, GPC, and Azure, plus incorporates a new dimension: prompt-optimization.
The panel warned: the opportunity is massive, but without observability, security, and strategy, the regrets will be real.
Rootly's open source SRE-skills-bench, which tests how LLMs handle real SRE scenarios, is now in Groq OpenBench, with steps to run it and get involved.
Page 1 of 3 · 26 articles