Skip to content

Incident Management

GitLab offers solutions for handling incidents in your applications and services, from setting up an alert with Prometheus, to receiving a notification via a monitoring tool like Slack, and automatically setting up Zoom calls with your support team.

Configuring incidents (ULTIMATE)

Introduced in GitLab Ultimate 11.11.

The Incident Management features can be enabled and disabled via your project's Settings > Operations > Incidents.

Incident Management Settings

Automatically create issues from alerts

GitLab issues can automatically be created as a result of an alert notification. An issue created this way will contain the error information to help you further debug it.

Issue templates

You can create your own issue templates that can be used within Incident Management.

To select your issue template for use within Incident Management:

  1. Visit your project's Settings > Operations > Incidents.
  2. Select the template from the Issue Template dropdown.

Alerting

GitLab can react to the alerts that your applications and services may be triggering by automatically creating issues, and alerting developers via email.

The emails will be sent to owners and maintainers of the project and will contain details on the alert as well as a link to see more information.

Prometheus alerts

Prometheus alerts can be set up in both:

Alert endpoint

GitLab can accept alerts from any source via a generic webhook receiver. When you set up the generic alerts integration, a unique endpoint will be created which can receive a payload in JSON format.

Read more on setting this up, including how to customize the payload.

Recovery alerts

GitLab can automatically close issues that have been automatically created when you receive notification that the alert is resolved.

Embedded metrics

Metrics can be embedded anywhere where GitLab Markdown is used, for example, descriptions and comments on issues and merge requests.

This can be useful for when you're sharing metrics, such as for discussing an incident or performance issues, so you can output the dashboard directly into any issue, merge request, epic, or any other Markdown text field in GitLab by simply copying and pasting the link to the metrics dashboard.

TIP: Tip: Both GitLab-hosted and Grafana metrics can also be embedded in issue templates.

GitLab-hosted metrics

Learn how to embed GitLab hosted metric charts.

Context menu

From each of the embedded metrics panels, you can access more details about the data you are viewing from a context menu.

You can access the context menu by clicking the {ellipsis_v} More actions dropdown box above the upper right corner of the panel:

The options are:

View logs

This can be useful if you are triaging an application incident and need to explore logs from across your application. It also helps you to understand what is affecting your application's performance and quickly resolve any problems.

Download CSV

Data from embedded charts can be downloaded as CSV.

Grafana metrics

Learn how to embed Grafana hosted metric charts.

Slack integration

Slack slash commands allow you to control GitLab and view content right inside Slack, without having to leave it.

Learn how to set up Slack slash commands and how to use them.

Slash commands

Please refer to a list of available slash commands and associated descriptions.

Zoom in issues

In order to communicate synchronously for incidents management, GitLab allows you to associate a Zoom meeting with an issue. Once you start a Zoom call for a fire-fight, you need a way to associate the conference call with an issue, so that your team members can join swiftly without requesting a link.

Read more how to add or remove a zoom meeting.

Configuring Incidents

Incident Management features can be easily enabled & disabled via the Project settings page. Head to Project -> Settings -> Operations -> Incidents.

Auto-creation

You can automatically create GitLab issues from an Alert notification. Issues created this way contain error information to help you debug the error. Appropriately configured alerts include an embedded chart for the query corresponding to the alert.