---
title: Alerting
description: The path from a failed check to a message in your inbox — confirmation, suppression, fan-out and retries.
sidebar:
  label: Overview
---

{/* Source of truth: packages/shared/src/notifications.ts (events),
    packages/api/src/worker/notification-fan-out.service.ts (suppression and
    fan-out), packages/api/src/monitors/... (confirmation). */}

A failed check does not send a message. Several things happen first, and knowing
the order explains most "why didn't I get an alert?" questions.

## The path

1. **A check fails**

    The probe records a failure with an [error code](/monitors/check-errors).
    Any configured `retries` have already been used up.

2. **Confirmation**

    Glowo waits for `confirmationsRequired` consecutive failures **and**
    `minFailureDurationMs` to elapse. Only then does the monitor become `DOWN`.
    See [Confirmations and flapping](/monitors/confirmations).

3. **Suppression**

    The status change is dropped if the monitor is inside a
    [maintenance window](/status-pages/maintenance-windows), or if the alert was
    recently [acknowledged](/alerting/acknowledging).

4. **Fan-out**

    Every channel subscribed to that event, and in scope for that monitor, gets
    a delivery. Channels are independent — one failing does not stop the others.

5. **Delivery and retry**

    Each delivery is recorded durably and retried on failure, with a per-channel
    circuit breaker. See [Delivery history](/alerting/delivery-history).

## Events you can subscribe to

| Event | Fires when |
| --- | --- |
| `MONITOR_DOWN` | A monitor is confirmed down |
| `MONITOR_UP` | A monitor recovers |
| `MONITOR_DEGRADED` | A `warn` assertion or a response-time threshold trips |
| `MONITOR_CHANGED` | A [DNS Diff](/monitors/dns-diff) answer changed |
| `CERTIFICATE_EXPIRING` | A TLS certificate is approaching expiry |
| `INCIDENT_CREATED` | An incident is opened on a status page |

Each [channel](/alerting/channels) subscribes to at least one of these. A
channel with only `MONITOR_DOWN` will never tell you the service came back —
a common and frustrating misconfiguration.

## Why you might not get an alert

In rough order of likelihood:

1. **The failure is not confirmed yet.** With the defaults, a monitor needs two
   consecutive failures and 60 seconds before it is `DOWN`.
2. **A maintenance window is active** for that monitor.
3. **Someone already acknowledged** the alert.
4. **The channel is not subscribed to that event.**
5. **The channel is scoped** to specific monitors or projects, and this monitor
   is not among them.
6. **The channel is disabled.**
7. **Delivery is failing** — check [delivery history](/alerting/delivery-history)
   for the channel.

:::note
Disabling a channel stops future pages, but deliveries already queued for it may
still go out. Disable means "stop paging me", not "cancel what is in flight".
:::

## A sane starting setup

- **Email** to the team, subscribed to `MONITOR_DOWN` and `MONITOR_UP`.
- **Slack or Discord** to your engineering channel, same two events plus
  `MONITOR_DEGRADED`.
- **SMS or PagerDuty** on `MONITOR_DOWN` only, scoped to your genuinely
  critical monitors.

The scoping in that last line is the important part. A phone that buzzes for
every monitor gets silenced, and then it is not a monitoring system any more.