Wazuh 5.0 monitor notifies the same burst every minute, and alerts on spread-out events

This note follows on from the Wazuh 5.0 migration check. There, under R2, we found that a 4.x rule with frequency and timeframe has no drop-in replacement on 5.0.0-rc1, and that a bucket-level Alerting monitor grouped by source.ip is the closest working path. This page covers two things that happen once that monitor runs: false alarms from a window wider than the rule you meant, and repeated notifications for one burst.

IDX=http://localhost:9200   # your indexer. Ours had the security plugin off: plain HTTP, no login.
CURL="curl -s"              # normal install: CURL="curl -sk -u admin:YOUR_PASSWORD" and IDX=https://...

$CURL "$IDX/_plugins/_alerting/monitors/MONITOR_ID" | python3 -m json.tool \
  | grep -E '"name"|"from"|"interval"|"unit"|per_alert|per_execution|actionable_alerts|"NEW"|"DEDUPED"|throttle_enabled'

$CURL "$IDX/_plugins/_alerting/monitors/alerts?size=200" | python3 -c '
import json,sys
for a in json.load(sys.stdin).get("alerts",[]):
    print(a["id"], a["state"], a["start_time"], a["last_notification_time"],
          (a.get("agg_alert_content") or {}).get("bucket_keys"))'

$CURL "$IDX/_cluster/settings?include_defaults=true&flat_settings=true" | python3 -m json.tool | grep host_deny_list
# and in the indexer log (ours: docker logs of the indexer container)
grep "host_deny_list" /path/to/indexer.log

Why it happens

Why it alerts on spread-out events

A bucket-level monitor does not count a sliding window the way a 4.x frequency / timeframe rule does. On each run (here once a minute) it runs one search over the last 3 minutes and counts events per source.ip. Anything that fits inside 3 minutes can be counted in full by one run.

Take 8 failures from one IP, about 21.4 seconds apart, over 150 seconds. No 120-second stretch holds more than 6 of them, so a rule meant as “more than 7 in 2 minutes” should stay silent. All 8 fit in 180 seconds, so the run that lands just after the last one sees 8 and alerts. In our test it did, in 3 of 3 runs. The window, not the trigger, decides what you are counting.

A window of 2 minutes avoids that case but misses others. In earlier runs (12 October, indexer heap 1 GB), now-2m missed 8 failures spread over 90 seconds and over 110 seconds, although both have 8 inside 120 seconds: no single run saw more than 7. That monitor ran once a minute. now-3m caught both. The migration check has the full table. For what changes when a now-2m monitor runs every 30 seconds or every second, see a 2-minute window with a shorter run interval.

Why one burst is notified again and again

A short burst stays inside a 3-minute window for three runs in a row. On the first run the alert is new. On the next runs the condition is still true, so the monitor keeps the same alert and marks it deduped. When a run no longer sees the burst, the alert is completed and disappears from the alert list (alert history is off by default on 5.0.0-rc1: plugins.alerting.alert_history_enabled is false in _cluster/settings?include_defaults=true).

Each action has an execution policy that decides which of those it reacts to. This is what each one did with one 16-second burst:

So the repeats are not a bug in the alert itself. There was one alert per burst. The action sent it again because its policy includes deduped alerts.

The monitor we used

Created through the Alerting API (POST /_plugins/_alerting/monitors), not through the dashboard. It counts every event with a source.ip in wazuh-events-v5-* over the last 3 minutes and fires on more than 7 per IP:

{
  "type": "monitor",
  "monitor_type": "bucket_level_monitor",
  "name": "brain50-ssh-bf-now3m",
  "enabled": true,
  "schedule": { "period": { "interval": 1, "unit": "MINUTES" } },
  "inputs": [ { "search": {
    "indices": [ "wazuh-events-v5-*" ],
    "query": {
      "size": 0,
      "query": { "bool": { "filter": [ { "range": { "@timestamp": { "gte": "now-3m" } } } ] } },
      "aggregations": { "by_src": { "terms": { "field": "source.ip", "size": 100 } } }
    }
  } } ],
  "triggers": [ { "bucket_level_trigger": {
    "name": "count-gt-7",
    "severity": "1",
    "condition": {
      "buckets_path": { "_count": "_count" },
      "parent_bucket_path": "by_src",
      "script": { "source": "params._count > 7", "lang": "painless" }
    },
    "actions": []
  } } ]
}

For the false-alarm test, that is the whole monitor: no actions. For the notification test, we used the same monitor (renamed brain50-ssh-bf-now3m-dup) with four webhook actions, all pointing to one listener in the lab, all with "throttle_enabled": false. They differed only in action_execution_policy. The one that sent a single notification per burst:

{
  "name": "hook-perAlert-NEW",
  "destination_id": "brain50-hook",
  "message_template": { "source": "A=perAlert-NEW period_end={{ctx.periodEnd}} new=[{{#ctx.newAlerts}}{{bucket_keys}};{{/ctx.newAlerts}}] dedup=[{{#ctx.dedupedAlerts}}{{bucket_keys}};{{/ctx.dedupedAlerts}}] completed=[{{#ctx.completedAlerts}}{{bucket_keys}};{{/ctx.completedAlerts}}]", "lang": "mustache" },
  "throttle_enabled": false,
  "subject_template": { "source": "perAlert-NEW", "lang": "mustache" },
  "action_execution_policy": {
    "action_execution_scope": {
      "per_alert": { "actionable_alerts": [ "NEW" ] }
    }
  }
}

The message template lists which bucket keys were new, deduped or completed in each notification. That is how we read the results below.

What we measured

Wazuh 5.0.0-rc1 (commit aff7a0b), our test containers: one manager and one indexer node, indexer heap 3 GB, security plugin off. Every event was a synthetic sshd failed-login line pushed into the manager's engine through its local ingest socket (engine-ingest-http.sock, POST /events/enriched). No real attacks, no agents. IPs are documentation addresses. Times are UTC.

Test 1: false alarm on 8 failures over 150 seconds (13 October 2026)

The monitor above, no actions, running every minute. Three rounds, each with a new IP. Each round sent 8 failed logins from one IP, about 21.4 seconds apart over 150 seconds, the last one about 15 seconds before a scheduled run. Each round also sent 7 failed logins from a second IP, below the threshold, as a control. We checked the alert list 6, 66 and 126 seconds after the run.

• It alerted in 3 of 3 rounds, at the first run after the last event. A 4.x rule with frequency="8" timeframe="120" is meant to fire only on 8 within 120 seconds, and these series never had more than 6 within 120 seconds. We did not replay these events on 4.x.
• The false alert lasted one run. At the next run, the first events had slid out of the 3-minute window: 5 of the 8 were still inside it 66 seconds after the run, and 2 after 126 seconds. The alert was gone from the alert list, and with alert history off there was no record of it. With no action configured, nobody would have seen it unless they were looking at that minute.
• The threshold held at its edge: the control IP with 7 events raised no alert at any check, in any round.

Test 2: notifications per burst, by execution policy (14 October 2026)

The same monitor with the four actions above. One burst of 10 failed logins from 198.51.100.52 over 16.4 seconds (02:37:57.682 to 02:38:14.096), plus 7 from a control IP 203.0.113.58. After the alert had cleared, a second burst of 10 from the same IP (02:43:07.587 to 02:43:22.973). We counted what the webhook listener actually received at each run.

• No duplicate alerts. Each burst produced one alert document. It kept the same id and start time on every run that still saw the burst; only its last notification time moved. The test stopped at 02:46, while the second alert was still active, so the second burst has no “completed” row.
• Repeated notifications are the default. The action with no execution policy behaved exactly like per_alert with ["DEDUPED", "NEW"]: three notifications for one 16-second burst.
• per_alert with ["NEW"] sent one notification per burst. The alert's own record agreed: that action's last execution time stayed at the first run while the other three moved on each run.
• A new burst after the alert cleared is a new alert with a new id, and every policy notified it. That is expected, not a duplicate.
• The control IP with 7 events raised no alert on any run.

How to configure it

1. One notification per burst (tested)

On every action of the trigger, set:

"action_execution_policy": {
  "action_execution_scope": {
    "per_alert": { "actionable_alerts": [ "NEW" ] }
  }
}

What you give up: no repeat while the burst is still inside the window, and no message when it ends. If you also want an “ended” message, per_execution sent one when the alert completed, but it also repeated on every run while the alert was active. We set these policies when creating the monitor. We did not test editing an existing monitor, through the API or the dashboard, and we did not check which policy the dashboard's monitor editor sets by default.

This limits notifications, not alerts. The alert still shows as active in the alert list on every run while the condition holds.

2. Fewer false alarms: pick the window you mean (trade-off, measured)

No action setting changes what the monitor counts. Only the window does, and both windows we tested were wrong in some cases, with the trigger _count > 7 and a 1-minute run interval:

The 150-second row is from 13 October with a 3 GB indexer heap. The other rows are from 12 October with a 1 GB heap; see the migration check. If a missed slow attack costs you more than a short-lived false alert, now-3m is the better of the two we tested. If false alerts cost more, use now-2m and accept that series spread over 90 to 120 seconds can be missed. Running it more often than once a minute catches some of those series, not all: see the shorter run interval test below.

Not tested: other window lengths, adding a filter on event.action, and the action throttle option. We left throttle off on every action. We cannot tell you how any of these behave on 5.0.0-rc1. (A run interval shorter than one minute was tested on 15 October; see below.)

3. Webhooks to private addresses are refused by default (tested)

In our first run of test 2 the listener sat on a private Docker address. The alert was ACTIVE on both runs we checked, and the alert recorded an execution time for all four actions, yet the listener received 0 notifications. The indexer log had:

Exception sending webhook message: message creation failed with status:Host of url is denied, based on plugin setting [notification.core.http.host_deny_list]

On 5.0.0-rc1 the default of opensearch.notifications.core.http.host_deny_list (read from _cluster/settings?include_defaults=true) blocks loopback, link-local and every private range, among them 127.0.0.0/8, 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16 and 100.64.0.0/10. If your webhook goes to a SOAR, an automation tool or a chat proxy inside your network, notifications fail silently: the alert looks normal and the error is only in the indexer log.

In the lab we cleared the list (PUT _cluster/settings with "opensearch.notifications.core.http.host_deny_list": []), and the second run delivered all 21 notifications in the table above. Do not do that in production. Allow only the address you need. We did not test a narrowed list.

A 2-minute window with a shorter run interval (added 15 October 2026)

A reader asked whether a now-2m monitor that runs more often than once a minute catches the 8 failures spread over 90 and 110 seconds that it missed above. We tested it on 15 October 2026 on Wazuh 5.0.0-rc1 (commit aff7a0b), our test containers. Short answer: more often helps, but it does not catch them all. Every 30 seconds, it caught the 90-second series in 7 of 12 rounds and the 110-second series in 0 of 12. Every second, it caught 5 of 5 and 4 of 5. What limits it is how late events become searchable, not how often the monitor runs.

The shortest schedule the Alerting API accepted

We sent the same monitor to POST /_plugins/_alerting/monitors with different schedules:

“Timezone” is the plugin's own wording. The shortest schedule is "period": { "interval": 1, "unit": "SECONDS" }; a cron schedule cannot go below one minute. Changing an existing monitor from 1 MINUTES to 30 SECONDS with PUT also worked and read back as 30 SECONDS. The 1-second schedule really runs every second: a monitor left on for about 25 seconds logged 25 Executing scheduled monitor lines in the indexer log, and in the test below it ran 978 times, 0.978 to 1.023 seconds apart. The 30-second schedule ran 67 times, 29.979 to 30.022 seconds apart. We created the 15-second monitor but did not run it.

What we ran

The monitor from above with "gte": "now-2m", the same _count > 7 trigger per source.ip and no actions: first on a 30-second schedule (12 rounds), then on a 1-second schedule (5 rounds). In each round four series ran side by side, each from a new documentation IP, with the events evenly spaced: 8 failures over 90 seconds, 8 over 110 seconds, 8 over 150 seconds (never more than 6 within 120 seconds, so it must not alert), and a control of 7 over 30 seconds (below the threshold, so it must not alert). Same synthetic sshd lines through the engine ingest socket as above. All 372 events in the 30-second phase and all 155 in the 1-second phase were accepted and indexed. We did not change the manager's flush setting: no override in its internal options file and none in the container environment.

“Time to searchable” runs from sending an event to the first search that found it. We searched about every 0.31 seconds, so each figure can be up to about half a second high.

• No false alarms. The 150-second series and the 7-event control raised no alert at either schedule: 0 of 17 rounds each.
• One alert per series that fired. Each IP that alerted had exactly one alert id: 7 IPs at 30 seconds, 9 at 1 second. No duplicate alerts.
• At 1 second, the alerts were short-lived. An alert for a 110-second series stayed in the alert list for about 5 to 9 seconds, and one for a 90-second series for about 20 to 28 seconds, before it completed and dropped out (alert history is off). These monitors had no actions, so we did not measure notifications. In test 2 above, an action with no execution policy notified again on every run while its alert stayed active; on a 1-second schedule that could mean many notifications per burst. If you shorten the schedule, set per_alert with ["NEW"] first.

Why a shorter interval is not enough

The monitor selects events by @timestamp, but it can only count an event once the event is searchable. On 5.0.0-rc1, analysisd sends events to the indexer in batches on a timer, analysisd.indexer_flush_interval, about 20 seconds by default (the setting and how to change it are in Wazuh 5.0 events appear about 20 seconds late). In this test new events became searchable in batches about 20 seconds apart (median spacing 20.01 s at the 30-second schedule, 19.95 s at the 1-second one), so each event took between about 1 and 21 seconds to show up.

A run sees all 8 events only between two moments: when the last event becomes searchable, and 120 seconds after the first event, when that first event drops out of now-2m. So:

catch window = 120 s − spread of the 8 events − time the last event took to become searchable

• 8 over 90 seconds: the window was 9.55 to 28.27 seconds long across our rounds. A run every second always landed in it. A run every 30 seconds did not always: runs fell at about :10.7 and :40.7 past each minute, while batches became searchable at about :03–:04, :23–:24 and :43–:44, so the next run came about 7, 17 or 27 seconds after the last event appeared. A round was missed when its window closed before that run. In one missed round the window was 21.23 seconds long: the last event became searchable at 14:37:43.447 UTC, and the next run was at 14:38:10.694, 27 seconds later.
• 8 over 110 seconds: the window is at most about 10 seconds, minus the last event's wait. In 8 of the 17 rounds it was negative, so no schedule could have caught those rounds. The 1-second miss was one of them: its last event took 17.75 seconds to become searchable, and the window was −7.60 seconds. Where the window was positive it was 2.10 to 9.14 seconds long; at 30 seconds none was longer than 5.25 seconds, shorter than the shortest wait for the next run, about 7 seconds.
• In all 51 rounds that were not controls (36 at 30 seconds, 15 at 1 second), “a run fell inside the catch window” matched “the monitor alerted”.

The 30-second results depend on where the monitor's runs fall relative to the flush cycle, which is set by when the monitor is enabled. We measured one such offset per schedule; with a different one, 7 of 12 would be a different number.

Not tested: lowering the flush interval together with a short schedule. The formula above suggests a shorter flush leaves a wider window, but that is arithmetic, not a measurement; we cannot tell you how it behaves on 5.0.0-rc1. Not measured: whether the Wazuh Dashboard monitor editor offers a schedule this short (we used only the REST API), the CPU and load cost of a 1-second monitor, other offsets between schedule and flush cycle, other flush values, heavy event load, real agents, and more than one indexer node.

How we counted what each run saw

With alert history off, a run that sees 7 events leaves no record. So we added a second trigger to the same monitor, _count > 0 (severity 5, no actions), and polled the alert list without pause: its alert carries the count that run saw for each IP. Both triggers are evaluated on the same search result, so the _count > 7 trigger was unchanged. Run times are from the indexer log. We checked each run's count against one rebuilt from our own searches: no difference at 30 seconds; at 1 second, 4 runs in 3 rounds differed, each time because the monitor found a new batch a fraction of a second before our search did. The table uses the monitor's own counts.