The hidden cost of vibe coding an incident bot

The weekend pitch

Vibe coding. You’ve heard it everywhere. Naturally, I’ve heard from plenty of people that they are going to “save money” by vibe coding their own incident bot. The pitch is always similar: a Slack app, a couple of slash commands, a hook into Jira. Claude can do it in a few hours. Why pay for something you can build before Monday?

I won’t argue with the first part. You can build something functional that quickly now.

I know because I’ve built incident bots twice, at two previous companies, long before vibe coding existed. The first bot was a lightweight piece of glue, and the second was a real product. We wrote it by hand, rewrote it by hand, and eventually started vibe coding add-ons. After years of tuning, it fit our incident process like a glove.

And then we replaced it with a vendor.

No, the bot didn’t fall over or have an outage. It was actually extremely reliable. This is the story of why a tool that worked still wasn’t worth owning, and what I’d warn anyone who’s considering building their own.

Disclaimer: You’re reading this on Rootly’s blog, and obviously Rootly sells incident tooling. Read this with that in mind, but know that the lessons hold whether you use Rootly or not.

Bot one: the glue

The first bot barely deserved the name. It tied together three tools we already used. Someone declared an incident in Slack using a basic Jira form, the bot opened a Jira ticket, paged the Incident Coordinator (me and only me), and sent the stakeholder emails through Gmail (which also showed up in Slack a few minutes later). That was most of it.

It was also a huge improvement over what came before. Before the bot, incidents were not logged at all; the only place they lived was in a shared Slack channel (and this was before threads existed!). After the bot, everything felt more organized. Responders stopped skipping steps because there were fewer steps to skip, and incidents became searchable in their own dedicated Jira project.

However, the bot taught me something I didn’t appreciate until later. An integration is a promise to keep up with three other companies’ roadmaps. Neither Atlassian, Slack, nor Google knew our bot existed, and none of them asked before changing things. An API version got retired, an auth flow changed, a field moved, an email bounced. Each change was small. Each one landed on me.

None of that made the bot a mistake. For the size of the team and the price of the tools, it was the right call. But I learned early that the build is the cheap part and the upkeep never ends.

Bot two: built by hand, twice

At another company, we went much further. This bot was a real application with its own codebase, its own deployments, and a list of feature requests that never got shorter.

It was initially written by hand (the horror!) before incident bots were something you could buy off the shelf. It handled the basics well. Declare, page, track, communicate, resolve. Over time, it picked up the habits of our process. New incident roles showed up as new commands. New severity rules became new logic. We designed our own retrospective template and hard coded it.

Eventually the code had grown in too many directions at once, so we modernized it by hand (again, pre-AI). That rewrite took several sprints of engineering time, but it was worth it. The new version was cleaner, faster to change, and easier for new people to understand. The Slack bot had cleaner, friendlier messaging (emojis ftw) that fit with our company culture. Metrics were logged in an AWS database and available to anyone at the company via Looker. Paging was integrated with the company directory, so service ownership was kept in sync automatically (unless there was another re-org).

Looking back, the rewrite is when I should have noticed what we’d signed up for. Nobody rewrites a weekend project; you rewrite software that people depend on. We weren’t maintaining a script anymore. We were running a product with one customer, our own incident process, and that customer was never going to stop asking for things.

Then features got cheap

Then AI coding tools took off, and everything sped up.

I remember the first feature we shipped this way. Someone asked for a small change in how the bot summarized an incident. A year earlier, that request would have sat in the backlog for a sprint or two. This time it was working the same afternoon.

For a while it felt like we had cheated. Every rough edge in our process could become a feature. A responder complained about a manual step on Tuesday, and by Wednesday the step was gone. The backlog that had been growing for years started to shrink.

After enough of that, the bot matched how we ran incidents almost exactly. Nobody had to bend the process to fit the tool. The tool bent to fit us. If you had asked me then whether we’d ever buy something off the shelf, I’d have laughed. Why would we give up a perfect fit that we have complete control over?

Here’s what I didn’t see while it was happening: every feature we added in an afternoon was a feature we’d own for years. AI made the building faster, but it didn’t make any of that code cheaper to maintain.

Why we left a perfect fit

Nothing fell over, we just started to add things up.

We sat down to plan the next year of work on the bot, and the list was longer than any year before it. Some of it was features people wanted, sure. Most of it was upkeep. Dependency upgrades. Security patches. Changes to keep up with Slack and the other tools we connected to. Work to make the bot hold up if the systems around it went down. Documentation, so the next person could maintain it. Meanwhile, the engineers who wrote the original code kept leaving the company, and their knowledge went with them.

Then we asked a harder question: who was doing all of this? The answer was a few people whose real job was commanding incidents. Every hour we spent on the bot was an hour we didn’t spend on the process the bot existed to support. The same people who were on-call for incidents were also on-call for incident infrastructure. Our plan for a production incident landing while the bot was down? Cross our fingers and hope it didn’t happen.

Then it clicked. We had built a second product. It had users, a roadmap, bugs, and a small team keeping it alive. And our business was running incidents well, not building the tools for it.

We wanted somebody else to own the roadmap, worry about reliability, and be on-call for it. We wanted to configure automation in a UI instead of shipping code. We wanted new incident and on-call features to show up without us building them.

So we bought one off the shelf. It wasn’t painless. We gave up some of the perfect fit, and a few people missed features only our bot had. But we got our time back, and we spent it where it mattered.

What vibe coding doesn’t fix

I’m a fan of AI coding tools. I use them every day. But they solve the problem that was already the easiest one. Some costs to consider:

Your bot has to work during your worst hour. Incident tooling gets its heaviest use when everything else is failing. If the bot runs on the same infrastructure that’s down, it goes down with it. Designing around that takes real engineering. A demo on a quiet Tuesday proves nothing.

Someone owns it forever. Code you ship in an afternoon still needs patches, upgrades, and fixes when an upstream API changes. That work lands on a person, and that person already has a day job.

The knowledge leaves with people. Homegrown tools live in one or two heads. When those people change teams, you’re maintaining code nobody fully understands. AI-written code can make this worse, because sometimes nobody fully understood it on the day it merged.

The questions keep coming. Leadership wants metrics. Auditors want timelines and access logs. Every one of those asks is another feature to build and defend.

None of these costs show up in the weekend estimate. All of them show up in year two.

Bigger teams did the same math

My story isn’t unusual. Some well-known engineering teams built their own incident tooling, and several of them have since moved to Rootly. Their reasons will sound familiar.

Replit built an internal Slack bot that opened incidents and pulled messages into a timeline. It ran on Replit’s own infrastructure, so a bad enough outage could take the bot down along with everything else. It also had no clear owner, so fixes happened whenever someone had time. “Once more than one incident was active, the old setup started working against us,” said Matthew Iselin, an engineering manager for SRE and security at Replit.

Webflow ran incidents on a few in-house bots and basic tooling, and running them was a side job for a couple of engineering managers. Over time, that setup stopped matching the scale and configurability the business needed. As Colin FitzGerald, Webflow’s problem and incident manager, put it, “We had reached the point where basic bots and ad hoc workflows were no longer enough.”

Lucidworks built its own tool early on. It created an incident Slack channel, added people to it, and connected to PagerDuty. Dave Bastomski, Lucidworks’ senior director of technical support, said “the tool we built just wasn’t reliable.” It would glitch or crash, and they didn’t want to train people to depend on something unstable. Maintaining it also wasn’t how they wanted to spend their engineers’ time. They moved to a vendor, and later to Rootly.

Nobody said their bot couldn’t open an incident. They talked about who owned it, what happened when the builder left, whether it would stay up in a real outage, and the next thing someone would ask it to do. None of that is factored into a weekend of vibe coding.

Before you open your editor

I’m not telling you never to build. For a small team with simple needs, a little glue can be the right answer.

But before you start, answer these honestly:

Who fixes the bot when it breaks in the middle of an incident?
Does it keep working when the rest of your stack doesn’t?
Who owns it a year from now, after the builder moves on?
Is building an incident tool the best use of your team’s next hundred hours?

If you have solid answers, go build. If your answers sound like “me,” “probably,” and “we’ll figure it out,” you’re not starting a weekend project. You’re starting a second product.

I loved that second bot. I’d defend most of what we built. But the goal was never a great bot. The goal was great incident response, and it took us years to see the bot had quietly become a job of its own.

Vibe coding makes it easy to start, but it doesn’t make it cheaper to own.

If you’d rather own your incident process than your incident tooling, book a demo and see what Rootly can do for you (and if you need help improving your incident process, shoot me a DM).