Auto Create Incidents from Any Alert

πŸ“Ÿ Auto Create Incidents from Any Alert

We are launching the ability to automatically create incidents from any alert source via Workflows.

  • Monitoring tools - Datadog, Grafana, Nobl9, etc.
  • Paging and alerting tools - PagerDuty, Opsgenie, VictorOps, etc.
  • Ticketing and tracking tools - Zendesk, Asana, etc.

Customers can configure a Workflow to ingest specific alerts from various external tooling and set a task to automatically kick-off incidents, regardless of source. This will spin up an incident channel on Slack, invite responders, start a Zoom, and/or whatever was configured.

Why did we build this? Because incidents can stem from many different sources such as programmatically, human detected, and more.

Rootly is the single aggregation point for all types of incidents and that can be created and managed consistently every time.

🌝 New & Improved

  • πŸ†• Design overhaul for timeline in Web platform to improve navigation and readability
  • πŸ†• Auto-assign Reporters and Assignee to Jira tickets in Workflows, e.g. {{ }}
  • πŸ†• Automatically record to cloud or local for Zoom bridges created with Workflows
  • πŸ†• Conditions in Workflows for set and unset (e.g. run Workflow A if Service is blank)
  • πŸ’… Ability to unsubscribe from status page updates
  • πŸ’… Datadog snapshots and dashboards posted into the incident channel by Rootly link back to Datadog
  • πŸ’… Running /incident list displays incident titles alongside their status
  • πŸ’… UI improvements to

Terraform Provider for Rootly

πŸ§ͺ Terraform Provider for Rootly

We are excited to introduce our long- awaited Terraform provider making infrastructure as code effortless.

Engineering teams can now manage Rootly directly in code making distributed ownership of their service ownership even easier.

No more needing to click into the Web platform to manage configurations, example use cases:

  • Manage teams and roles
  • Manage severities, incident causes, types, etc.
  • Import infrastructure such as services and functionalities

Our Terraform provider can be found here and documentation here.

In addition, we also have our Rootly API available, learn more here.

🌝 New & Improved

  • πŸ†• Ability to configure preheader text in send email task sent via Workflows
  • πŸ†• Ability to turn on/off broadcasting of incident updates in channel vs. just threaded message, found in Integration > Slack
  • πŸ†• Templates in Workflows can be accessed as a pop-up
  • πŸ’… Updated Jira and Confluence scopes to leverage the latest versions
  • πŸ’… Additional guidance to Bookmarks task and PagerDuty task in Workflows
  • πŸ’… Reliability enhancements across how we handle backend Slack interactions
  • πŸ’… Updated /incident help with /incident support and /incident convert command and guidance
  • πŸ’… If roles are being reassigned we no longer notify the channel reducing noise
  • πŸ’… Improved descriptions around Kind of incidents (e.g. Test / Training, Incidents, Scheduled Maintenance) in Workflow conditions
  • πŸ’… Improved internal error reporting to more easily identify customer specific issues
  • πŸ› Fixed issue where users could not unassign roles after the incident was resolved in Web
  • πŸ› Fixed issue where active incident count on Web dashboard was incorrect
  • πŸ› Fixed issue where some users occasionally were not being invited via PagerDuty
  • πŸ› Fixed issue where searched items in Web occasionally did not send users to the right incident

Get Support From Rootly in Slack

πŸͺ‚ Get Support From Rootly in Slack

We are making accessing Rootly support and engineering team even easier.

We are waving goodbye to support portals, Zendesk, and other methods.

Anyone from within Slack can run /incident support to file a ticket (report bug, get help, etc.) with Rootly. As always, you'll still have access to us via Slack Connect channels.

If urgent, we trust our customers to even page our engineering team and we'll wake up.

Why did we do this? This is part of our continued investment to ensure our customers have a successful implementation day 1 and day 365+. A few other helpful tools.

  • /incident tutorial - guided and customized tutorials to teach your users, learn more.
  • /incident test - create test incidents to get hands on experience with Rootly without false alarms
  • /incident convert - channels you might forget to create with Rootly can be turned into Rootly incidents, learn more.

🌝 New & Improved

  • πŸ†• One-click re-integrate Slack to upgrade permissions without needing to uninstall first
  • πŸ†• Export action items to CSV
  • πŸ†• Bookmark automatically created whenever a new integration is configured
  • πŸ†• Support for preheader text in send email Workflow task
  • πŸ†• URL that automatically opens up incident creation on Web:
  • πŸ’… Ability to hide manual Workflows drop-down as part of incident creation wizard on Slack
  • πŸ’… User prompts to upgrade Slack permissions to unlock new features
  • πŸ’… Deleting all roles will hide role assignment on Slack
  • πŸ’… Added Twitter links to Changelog page, keep up with our latest updates!
  • πŸ’… Deleting and editing Services, Types, Functionalities, and other conditions now require two less clicks
  • πŸ› Fixed issue where postmortems could not be bookmarked
  • πŸ› Fixed issue where had confusing Add to Slack links

Bookmark Anything in Incident Slack Channel

πŸ”– Bookmark Anything in Incident Slack Channel

We are introducing a new Workflow that allows you to bookmark any link, integration, or resource to the incident Slack channel.

Customers can now append the most critical information at the top of incident channels making it immediately accessible.

Built on top of our Workflows engine, not only are there powerful defaults out of the box but can be customized to suit any organization.

  • Have different bookmarks for different types of incidents (e.g. Security playbook vs. Infrastructure playbook)
  • Specify Title, Link, and Emoji
  • Out of the box incident dashboard and relevant integrations are bookmarked

So, what can you bookmark automatically? Pretty much anything, here are a few examples.

  • Integrations (e.g. Jira issues, Zoom rooms, Confluence postmortems, PagerDuty on-call schedule, etc.)
  • Internal resources (e.g. Datadog dashboards, etc.)
  • Playbooks / Runbooks (e.g. Restarting the database, Cloud provider outage, etc.)
  • ...endless possibilities!

🌝 New & Improved

  • πŸ†• The Rootly team will be at SREcon San Francisco from March 14-16 at Booth 111, come say πŸ‘‹
  • πŸ†• Condition in Workflows to specify Test / Training incidents that enable ability to build customized training
  • πŸ’… Canceled incidents do not show up on Dashboard anymore
  • πŸ’… Revamped /incident update modal
  • πŸ’… is more mobile responsive and friendly
  • πŸ› Fixed issue where Rootly status page updates were formatted incorrectly

Convert Existing Channels to Rootly Incidents

🧞 Convert Existing Channels to Rootly Incidents

Rootly provides the most comprehensive suite of onboarding tools to help companies rapidly adopt better incident management.

As with the introduction of any tool, you won't always remember to use Rootly for new incidents due to muscle memory.

For the times you forget, we are introducing /incident convert that allows you to turn existing incident channels into Rootly incidents.

  • Continue collaborating on incidents without losing context or stopping your conversation
  • Incident history preserved and no need to manually backfill into Rootly
  • Link to existing incidents if an on-going incident had already been created
  • Prompts to fill in incident details as if you were just creating a new incident (e.g. Severity)
  • Supports private incident channels

🌝 New & Improved

  • πŸ†• New coat on paint on with shiny new logos, testimonials, and product features
  • πŸ†• Reordering teams in Web reflect the order displayed in Slack create and update incident wizard
  • πŸ†• Ability to reorder components on Rootly status page
  • πŸ†• Ability to resend invites to Rootly with one click in Organization Members
  • πŸ†• New incident variable for {{ incident.roles }}
  • πŸ†• Integrations when incidents are created are shown as hyperlinks
  • πŸ’… Increased number of times we refresh and check for new Slack channels, reminder you can always go to Integration > Slack > Refresh Channels without waiting
  • πŸ’… Weekly summary emails are now sent Monday at noon instead of Friday end of day, enjoy your weekend!
  • πŸ’… Playbooks are displayed in two columns instead of one to reduce visual clutter
  • πŸ’… Continuing major refactoring internally to ensure Slack commands are even more resilient
  • πŸ’… Removed /incident connect step in /incident tutorial as all new users are automatically prompted to connect
  • πŸ› Fixed issue where removing labels from Jira did not save for some users
  • πŸ› Fixed issue where selecting State in Linear Workflow task did not populate results for some users
  • πŸ› Fixed issue where clicking save on Linear in Integration gave users an error page despite saving correctly
  • πŸ› Fixed issue where postmortem download to PDF was stalling
  • πŸ› Fixed issue where we displayed a duplicate Workflow task for inviting on-call users to Slack channels

Incident Preview

πŸ” Incident Previews

The vast majority of customers have a global Slack #incident / #outages / #warroom channel.

This is where all incidents get posted for broader visibility.

At a quick glance, we are making it now even easier to know whether or not its important to various stakeholders.

  • Dynamic based on impact of incidents to avoid information overload (e.g. if no Service impacted then Service field won't display)
  • Shows any impact described at incident creation (e.g. SEV, Teams, Environments, etc.)
  • UI easily digestible to know if you should join the incident
  • Bold CTA guiding users to join incident channel if relevant

🌝 New & Improved

  • πŸ†• Ability to duplicate Workflows for faster repetitive automations
  • πŸ†• Custom date range filters instead of using defaults like Today, Last 7 Days, etc.
  • πŸ†• Updates to the incident (via /incident update) now get posted as a thread instead of new message in channel for easier access and to reduce noise
  • πŸ†• GCP added as 3rd party component in status pages
  • πŸ†• Assigning roles from PagerDuty on-call rotation supports override policies
  • πŸ†• Roles can now be assigned to the person creating the incident via Workflows
  • πŸ†• Workflow task to Update Incident (e.g. if incident updated to Security then update visibility to Private)
  • πŸ’… If GitHub integration returns zero commits we still let you know there is 0 commits
  • πŸ’… GitHub repos in timeline are now hyperlinked for easier access
  • πŸ’… After your account is connected with Slack we redirect you back to Rootly
  • πŸ’… Continued infrastructure enhancements to how we handle our Slack slash commands
  • πŸ› Fixed issue where emails Workflow did not respect defined formatting
  • πŸ› Fixed issue where Google Meet Workflow task had a typo

Custom Role-Based Access Control (RBAC)

πŸ” Custom Role-Based Access Control (RBAC)

Rootly is already the most secure and enterprise-ready incident management platform on the market by a margin.

Today, we are taking that further by introducing role-based access control (RBAC) to our customers.

Why? Not all incident data or access is as simple as having Admin and User.

For example, create a Security role only for your InfoSec team or an Observer role for read-only access to give customer support.

  • Unlimited custom role creation with CRUD definitions
  • Granular permissions (e.g. configurations, incident data, Workflows, integrations, etc.)
  • Easily assign existing and new members new roles
  • Limit access to both Slack app (e.g. /incident new) and Web UI
  • Control who can access Private incidents (e.g. data breach)
  • Out of the box popular roles pre-defined and ready to use

🌝 New & Improved

  • πŸ†• Ability to edit timestamps of events added to the incident timeline
  • πŸ†• Send Slack message Workflow task supports pinging individual users instead of just channels
  • πŸ†• We want to encourage users to keep opening incidents, now they can be marked as Canceled
  • πŸ†• Workflow condition status for Canceled incidents
  • πŸ’… Made it easier to find the correct task type for Jira projects inside of Workflows
  • πŸ’… More descriptive error messaging at sign up for incorrect user actions
  • πŸ’… Infrastructure improvements on how we handle Slack commands internally, better reliability and performance
  • πŸ’… Welcome message to greet you when you log into Rootly on Web
  • πŸ’… Private incident alerts are always muted by default
  • πŸ› Fixed issue where all Asana projects did not load for larger organizations
  • πŸ› Fixed issue where invite to incident channel Workflow template had repeat and delay values set
  • πŸ› Fixed issue where deleting incidents in Web did not immediately get reflected in the UI

Subscribe to Incidents

πŸ”” Subscribe to Incidents

We are introducing the ability to subscribe to incidents because staying in the loop is critical.

But different audiences want to be updated differently. A couple of examples.

  • Impacted Engineers - want to know every message sent because the details matter
  • Leadership - want to know when the incident is mitigated and resolved, exact details matter less

With the ability to subscribe to incidents, those that just want to follow along easily can do so.

  • Email notifications whenever the incident is Mitigated and Resolved
  • Anyone that enters the incident Slack channel automatically becomes a subscriber
  • Give your organization control to choose how they would like to be updated
  • Ability to subscribe or unsubscribe with a single click

🌝 New & Improved

  • πŸ†• Overview when incident Slack channel first created is automatically pinned for easier access
  • πŸ†• Ability to unpublish and edit already published postmortems inside of Rootly
  • πŸ†• Workflows run sequentially in 1-2-3 order instead of all at once
  • πŸ†• View in Linear button added next to any Action Items that are created
  • πŸ†• Linear integration in Workflows now support the ability to create incidents in a specific state (e.g. Todo)
  • πŸ’… Significantly reduced the number of clicks required to quickly edit configurations (e.g. Severity, Services, etc.) by redirecting users back to list view
  • πŸ’… New and standardized error pages for Web
  • πŸ’… UI improvements to postmortem tab in Rootly to make it clear when it's editable
  • πŸ’… New logos on Rootly homepage of customers we partner closely with, we'll periodically rotate them
  • πŸ’… New coat of paint for with a much more modern feel
  • πŸ’… Warning message indicating all other login methods will be disabled when SSO is enabled
  • πŸ› Fixed issue where hyperlink in Slack did not redirect you back to the status page after updating
  • πŸ› Fixed issue where images did not render properly inside of Rootly's native postmortem editor
  • πŸ› Fixed issue where searching for Jira project keys did not work inside of Workflows
  • πŸ› Fixed issue where search for Confluence space key did not work inside of Workflows

Workflow Templates

πŸ“œ Workflow Templates

We are excited to introduce Workflow Templates to help you get started with Rootly even faster.

This is just one of the ways we continue to a) provide the most powerful defaults out of the box but b) still allow for flexibility and configurations.

Customers run hundreds of thousands of Workflows and we templated some of the most common ones:

  • Send update email
  • Reminder to update status page
  • Send generic reminder to incident channel
  • Invite users and teams to Slack
  • Send SMS and call
  • Auto-archive the incident
  • Auto-resolve the incident
  • Notify teams on Slack and summarize the incident
  • Page PagerDuty/Opsgenie responders

Many more to come!

🌝 New & Improved

  • πŸ†• Action items are now linked on postmortems written in Confluence and Google Docs
  • πŸ†• Slack notification to notify any incident subscribers whenever a postmortem is published
  • πŸ†• Workflow Task that allows auto-publishing events to your status page
  • πŸ’… Made UI changes to to make reading more pleasurable
  • πŸ’… Edits to that better highlight our strict compliance and security standards
  • πŸ› Fixed issue where CSV export did not respect pre-selected filters
  • πŸ› Fixed issue where creating Rootly status pages did not let you set it to public by default
  • πŸ› Fixed issue where some users were unable to mark incident action items as done in Slack
  • πŸ› Fixed typos in Incident Type
  • πŸ› Fixed issue where private incidents did not automatically prompt users to go edit postmortem

Incident Variables and Snippets

βœ‚οΈ Incident Variables and Snippets

Incident variables such as {{ incident.severity }} are a way to quickly contextualize your communication without manual input.

We are excited to launch 100+ incident variables that span across Rootly and it's integrations (e.g. Jira, Google Doc, etc.).

They can be used in Workflows, incident title, postmortem templates, emails, and nearly everywhere in Rootly.

The use cases are endless, for example:

  • Summarize incident for leadership email {{ incident.summary }}
  • Rename Slack channel to align with Jira ticket number {{ incident.jira_issue_id }}
  • Indicate if incident Type is customer facing in postmortem title {{ incident.type }}
  • Send security team Slack message with incident channel link {{ incident.slack_channel_url }}

Incident variables and snippets are easily accessible via drop-down where available, no copy-paste required.

They are also dynamic in nature so you can set it and forget it.

🌝 New & Improved

  • πŸ†• Integrations such as Jira and Confluence links are now displayed at the top of incidents on Web for easier access
  • πŸ†• Mitigated and Resolved messages now get posted in incident channel, main incident notification channel, and in emails
  • πŸ†• Ability to search for organization members in
  • πŸ†• Total number of organizational members displayed now displayed
  • πŸ’… Invested in our backend infrastructure that makes editing content in Rootly snappier
  • πŸ’… Cleaned up native edit Postmortem UI
  • πŸ› Fixed issue where action items inside of Slack could not be marked as done
  • πŸ› Fixed issue where cookie consent over crowded the UI

Rootly helps you respond to incidents faster as a team and learn to prevent them in the future.