mapper_parsing_exception: the alert is in alerts.json but never reaches the dashboard

curl -sk -u '<user>:<password>' \
  "https://<indexer>:9200/wazuh-alerts-*,wazuh-archives-*/_mapping/field/data.ms-graph.status,data.ms-graph.appliedConditionalAccessPolicies?pretty"

grep -o 'failed to parse field \[[^]]*\]' /var/log/filebeat/filebeat* | sort | uniq -c | sort -rn

Why it happens

The manager writes the alert to alerts.json. Filebeat then sends it through an ingest pipeline into an index whose mapping comes from the Wazuh index template. The mapping allows one type per field path. When a document carries a different shape at that path, the indexer answers with mapper_parsing_exception and does not store the document. Because the alert is already in alerts.json, everything on the manager side looks normal.

Two things make a clash likely:

• The stock template maps many fields as keyword. In the 4.14.7 template we counted 609 leaf fields, 486 of them keyword, 409 of those under data.*. Directly under data there are 21 scalar fields, among them data.data, data.srcip, data.url and data.id. A log source that sends an object under one of those names can be rejected.
• The JSON decoder changes shape with the input. In wazuh-logtest on 4.14.7, an object is expanded into subfields, while an array, empty or not, is packed into a single string such as '[]'. We saw this on two integrations, Nextcloud and Office 365. So the same path can be an object in one event and a string in the next.

Case 1: Microsoft Graph sign-ins, data.ms-graph.status

The template maps data.ms-graph.status and data.ms-graph.appliedConditionalAccessPolicies as keyword. We checked the template at release tags v4.8.0, v4.10.0, v4.12.0, v4.14.0, v4.14.3, v4.14.7 and v4.14.8: the same in all seven. Alerts and incidents send status as a string such as "resolved". Sign-ins send it as an object, and send the policies as a list of objects. On a test indexer every sign-in was rejected, in both arrival orders:

failed to parse field [data.ms-graph.status] of type [keyword] ... Can't get text on a START_OBJECT

The error names only the first field it trips on. A sign-in with a Conditional Access policy fails on appliedConditionalAccessPolicies; one with an empty policy list fails on status. When we fixed status alone, sign-ins that had a policy applied were still rejected.

A related detail: the stock 4.14.7 ruleset (0995-microsoft-graph_rules.xml, 99 rules) has no rule for relationship: signIns. A sign-in only matches 99500 at level 0, so it reaches the indexer through the archives, or through alerts once you write your own sign-in rule.

Case 2: a field that is sometimes an object, sometimes a string, data.data

Nextcloud events can carry data as an object or as an array. Our three test lines differed only in that, and all three matched rule 88207. With the stock template, where data.data is keyword, the object shape fails with failed to parse field [data.data] of type [keyword]. A separate template that makes data.data an object accepts those, and rejects the string shape instead. Changing the template alone only moves which shape fails.

Fix

Case 1: move the object shape to a new field, in the pipeline

Keep the string status of alerts and incidents where it is, because dashboards and saved searches expect it, and move only the sign-in shapes. One script processor, placed right after the json processor, in both the alerts and the archives pipeline:

{"script": {
  "lang": "painless",
  "ignore_failure": true,
  "description": "ms-graph signIns fix",
  "source": "def d = ctx.get('data'); if (!(d instanceof Map)) { return; } def g = d.get('ms-graph'); if (!(g instanceof Map)) { return; } if (g.get('status') instanceof Map) { g.put('signInStatus', g.remove('status')); } def p = g.get('appliedConditionalAccessPolicies'); if (p instanceof List && !p.isEmpty() && p.get(0) instanceof Map) { g.put('conditionalAccessPolicies', g.remove('appliedConditionalAccessPolicies')); }"
}}

• ignore_failure: true: the stock pipelines end with on_failure: drop, so a processor that throws would drop the whole event. With this flag the worst case is an event indexed unchanged.
• d.get('ms-graph') rather than ctx.data?.ms-graph: Painless reads the hyphen as a minus sign, the dotted form does not compile, and the indexer refuses the whole pipeline. We hit this ourselves.

What we measured: every sign-in and the alert indexed, in both orders, in a new index and in an index that already existed, so the fix works on today's index. data.ms-graph.status stays keyword and still aggregates. The complete script, with verify, apply and rollback, is in our write-up on dev.to.

Case 2: a separate template plus a rename by type

Two changes that only work together. First, a separate legacy template with a higher order that keeps the field an object:

PUT _template/fixpack-data-data
{"order": 1,
 "index_patterns": ["wazuh-alerts-4.x-*"],
 "mappings": {"properties": {"data": {"properties": {"data": {"type": "object"}}}}}}

Second, one rename processor right after the json processor, which moves the field only when it arrives as a string, so the new field only ever receives strings:

{"rename": {"field": "data.data", "target_field": "data.data_raw",
            "ignore_missing": true, "if": "ctx.data?.data instanceof String"}}

What we measured: every order of object, "[]" and "[{..}]" indexed cleanly, 4 of 4 in two separate orders. The template alone, the rename alone, and a rename by location each still failed. A template only applies when an index is created, so this fix takes effect on the next daily index. Note the side effect: the rename moves every string data.data, from every source, into data.data_raw, so searches and dashboards that read data.data as a string need to follow it.