Allowlisting Canary9 probes

CDN and WAF bot management can mistake a monitoring probe for an unwanted bot. Here's why that happens, how Canary9 handles it when it does, and how to allowlist our checks with the most common providers.

Why bot management blocks monitoring probes

Products like Cloudflare Bot Management, Akamai Bot Manager, and AWS WAF Bot Control score incoming requests and challenge or block ones that look automated: no browser fingerprint, no cookies, a predictable interval, a non-browser User-Agent. A synthetic monitoring probe looks exactly like that by design, so it's common for these products to return a 403 or an interactive challenge page to a legitimate Canary9 check.

That's a false positive from your monitoring's point of view: the target is up, but the check can't reach it past the bot-management layer. Left unhandled, this shows up as phantom downtime: an outage alert for a site that's actually fine.

How Canary9 handles blocked checks

Canary9 recognizes the common bot-management block patterns (403 responses and interactive challenge pages from major CDNs) and records those checks with a Blocked result instead of a failure:

• Blocked checks render in violet in your uptime and response-time charts, distinct from both successful and failed checks.
• Blocked checks do not fire outage alerts and do not count against uptime, since they don't tell you anything about whether your service is actually up.

This keeps a misconfigured allowlist from paging your team, but it's still worth fixing: while checks are blocked, you have no real signal from that location, which is a monitoring gap even if it isn't an alert.

This applies to browser checks as well as HTTP checks. A browser check loads a challenge page successfully, since nothing about the page load itself fails, so we look at what actually rendered and record a Blocked result when it's a challenge or CAPTCHA interstitial rather than your site. A screenshot is still captured, so you can see exactly what the probe was served.

Allowlist by provider

The instructions below add an exception for the Canary9 User-Agent. They're accurate as of this writing, but bot-management UIs change, so if a step looks different in your dashboard, look for the equivalent "custom rule" or "exception" concept.

Cloudflare

1. In the Cloudflare dashboard, go to Security → WAF → Custom rules for the zone Canary9 monitors.
2. Create a rule that matches when User-Agent contains Canary9-Probe. Match on contains rather than an exact string: browser checks send Chrome's own User-Agent with the Canary9-Probe marker appended, so only a substring match catches both check types.
3. Set the action to Skip, and skip the remaining WAF rules and bot-management challenge you want the probe to bypass (typically "Bot Fight Mode" / "Super Bot Fight Mode" and any managed challenge rules).
4. Save and deploy the rule.

Akamai

1. In Akamai Bot Manager, create a bot definition (or client list) matching User-Agent containing Canary9-Probe.
2. Add the definition as an exception to the relevant bot category or detection rule so matching requests are categorized as an allowed/known bot rather than challenged.
3. Confirm the exception applies to the property/hostname Canary9 checks, then activate the change on the appropriate network (staging, then production).

AWS WAF

1. In the AWS WAF console, open the web ACL protecting the monitored resource (ALB, CloudFront distribution, or API Gateway) and find the Bot Control managed rule group.
2. Add a scope-down statement to the Bot Control rule group that excludes requests where the User-Agent header contains Canary9-Probe. A string-match condition on the User-Agent header is sufficient.
3. Alternatively, add a separate custom rule with higher priority than Bot Control that matches the same User-Agent condition and sets the action to Allow, so it short-circuits before Bot Control evaluates the request.
4. Save and deploy the web ACL.