How monitoring works

The loop

Every monitor runs on its interval. At each interval, one check runs from each region you selected. A monitor on a 1-minute interval with three regions makes three checks a minute.

Each check produces a result: up or down, how long it took, and the error if there was one. Those results are what the uptime figure and the charts are built from.

When an incident opens

This is the rule, exactly:

1. Results arrive from every region into one stream, in the order they arrive.
2. When the number of consecutive failures reaches the failure threshold, an incident opens and the alert goes out.
3. One success closes it, and the recovery alert goes out.

The important word is consecutive, and the important detail is that the count is not per region. With a threshold of 3, three failures from three different regions open an incident just as three from one region would.

That is deliberate. A failure seen from three places is better evidence of an outage than three failures from one, not worse — and requiring three in a row per region would quadruple the time to alert on a site that is genuinely down everywhere.

The practical consequence worth knowing: a monitor with many regions reaches its threshold faster, because results arrive more often.

What the threshold is for

One failed check is not evidence. Packets get dropped, a worker restarts, a load balancer moves a connection. A threshold of 1 will page you for all of those.

The default is 2, and that is a reasonable floor for most things. Raise it for anything you know to be flaky; the cost is that an outage takes longer to confirm.

Time to alert

Roughly: interval × failure threshold, plus the time the check itself takes.

A 1-minute interval with a threshold of 3 means about three minutes before you hear anything. If that is too slow, shorten the interval rather than lowering the threshold — you get the same speed with the same resistance to noise.

What does not follow this rule

• Heartbeat monitors wait to be pinged instead of going out to look. They have no regions, and their threshold is fixed at 1.
• Content change monitors never go down at all. They announce a change and nothing else.
• Server checks are pushed by the agent, not polled, and they need the threshold met on two consecutive reports.
• Latency alerts notify without changing status, so they never open an incident.