AI alert triage
Not every failed run is a problem a visitor would run into. Relvato AI triage reads each failed run and says whether it needs your attention or is most likely noise, with one sentence why. You decide how far it goes: a label only, a label plus holding back the alert, or setting the noise aside entirely. It's on the Pro plan and up, and off until you switch it on for a site.
What triage says about a failed run
Each failed run gets one of three verdicts. Needs attention: a real problem to look at. Likely noise: something the monitor reported that no real visitor runs into — a link hidden inside a cookie banner, an unfilled template placeholder, a quirk of how the monitor measured. Repeat: the same failure as the monitor's previous run, still open.
You see the verdict on the run page, in the failed run's card, and as a small label beside the status in the run lists. Hover it for the reason. A verdict decided by the AI says so (“AI: likely noise”) and carries its confidence; a repeat, and a failure Relvato always treats as needing attention, are decided by a rule without the AI.
The four settings
Pick one per site in Settings → General → Relvato AI triage. Each one includes everything before it.
Holding back and setting aside act only on the AI's own “likely noise” verdict given with high confidence. A medium or low confidence verdict, a repeat, or no verdict at all changes nothing: the failure alerts as it always did.
What is never held back or set aside
Checkout, payment, cart and order monitors, sign-in and account monitors, uptime, SSL, the domain monitors and every security monitor are never left to the AI. Their failures are always marked Needs attention and always alert — Relvato checks that before asking the AI and again before holding anything back. See Alert severity levels for which monitors are high or critical.
If the AI doesn't answer in time, is unavailable, or a site has reached its daily limit of 40 AI triages, the failure simply goes unlabelled and alerts as usual.
Bringing a failure back
A failure that was set aside is listed at the top of the site's Runs tab, under Set aside by Relvato AI, and its run page says so with the reason. If it's real, click Bring it back: it counts as a failure again in the bell, the digest and site health. No alert is re-sent. An assistant connected over MCP can do the same with the bring_back_run tool.
If the AI got it wrong, Report a false positive on the run tells Relvato's team, so the monitor itself gets fixed — that's how the rule gaps behind noise are found.
What is sent to Anthropic
To decide, Relvato sends the failed run's details to Anthropic, which provides the AI model: the error, what the monitor found, the failing steps, whether the monitor failed before, and the failure screenshot. Never your passwords, API keys, tokens, the test customer's sign-in or the activity log. Anthropic is listed with Relvato's other sub-processors on Trust.
Turn it on
Open the site, go to Settings → General → Relvato AI triage, pick a setting and click Save. It applies to failed runs from then on; earlier runs stay as they are. On the Free plan the settings are shown but can't be chosen.
Over MCP or the API, update_site_settings takes aiTriage: off, label, hold or aside, and get_run returns each run's verdict.
What each setting changes
| Setting | What Relvato AI does | Instant alert | Bell, digest, site health |
|---|---|---|---|
| Off | Nothing | As always | As always |
| Label each failure | Labels every failed run: needs attention, likely noise or repeat, with why | As always | As always |
| Label, and hold back alerts for likely noise | Labels, and holds back the instant alert of confident noise | Held back for confident noise | Still listed |
| Label, hold back, and set aside likely noise | Labels, holds back, and sets the confident noise aside | Held back for confident noise | Left out until you bring it back |
FAQ
Does the AI decide whether my site passed?
No. The monitor decides pass or fail; triage only labels a failure and, if you choose, holds back its alert or sets it aside. A set-aside failure is still a failure on its run page, worded “Set aside by Relvato AI”, never “fixed”.
Can a held-back alert hide a broken checkout?
No. Checkout, sign-in, uptime, SSL, domain and security failures are never held back or set aside, whatever the AI says.
Which plans include it?
Pro, Business and Agency. It's off for every site until you switch it on.