elastic/elasticsearch · error · IllegalArgumentException

unable to construct flow from document

Error message

unable to construct flow from document

What it means

CommunityIdProcessor.execute calls buildFlow, which returns null when source.ip, destination.ip, or both are absent, or when both iana_number and transport are missing. If ignore_missing is false (note: the default for CommunityIdProcessor is true, so this throw is opt-in), execute throws this message instead of passing the document through.

Source

Thrown at modules/ingest-common/src/main/java/org/elasticsearch/ingest/common/CommunityIdProcessor.java:148

        return ignoreMissing;
    }

    @Override
    public IngestDocument execute(IngestDocument document) throws Exception {
        String sourceIp = document.getFieldValue(sourceIpField, String.class, ignoreMissing);
        String destinationIp = document.getFieldValue(destinationIpField, String.class, ignoreMissing);
        Object ianaNumber = document.getFieldValue(ianaNumberField, Object.class, true);
        Supplier<Object> transport = () -> document.getFieldValue(transportField, Object.class, ignoreMissing);
        Supplier<Object> sourcePort = () -> document.getFieldValue(sourcePortField, Object.class, ignoreMissing);
        Supplier<Object> destinationPort = () -> document.getFieldValue(destinationPortField, Object.class, ignoreMissing);
        Object icmpType = document.getFieldValue(icmpTypeField, Object.class, true);
        Object icmpCode = document.getFieldValue(icmpCodeField, Object.class, true);
        Flow flow = buildFlow(sourceIp, destinationIp, ianaNumber, transport, sourcePort, destinationPort, icmpType, icmpCode);
        if (flow == null) {
            if (ignoreMissing) {
                return document;
            } else {
                throw new IllegalArgumentException("unable to construct flow from document");
            }
        }

        document.setFieldValue(targetField, flow.toCommunityId(seed));
        return document;
    }

    public static String apply(
        String sourceIpAddrString,
        String destIpAddrString,
        Object ianaNumber,
        Object transport,
        Object sourcePort,
        Object destinationPort,
        Object icmpType,
        Object icmpCode,
        int seed
    ) {

View on GitHub (pinned to db6a809a66)

Solutions

1. Confirm source.ip and destination.ip (or your configured field paths) are populated on every document before community_id runs.
2. If missing values are legitimate, leave ignore_missing at its default (true) — the processor will pass such documents through unchanged.
3. Add a mapping/ enrichment step that backfills network.iana_number or network.transport so buildFlow can resolve the protocol.
4. Pre-route documents lacking the required fields away from the community_id processor.

Example fix

// before — strict config, event with no IPs fails
//   { "community_id": { "ignore_missing": false } }
//
// after — tolerate events that lack a complete flow
//   { "community_id": { "ignore_missing": true } }

// All three are required to build a flow.
boolean canBuildFlow(IngestDocument d) {
    return d.hasField("source.ip") && d.hasField("destination.ip")
        && (d.hasField("network.iana_number") || d.hasField("network.transport"));
}

{
  "community_id": {
    "ignore_missing": true,
    "on_failure": [
      { "set": { "field": "ingest.error", "value": "community-id-missing-flow" } },
      { "redirect": { "pipeline": "quarantine" } }
    ]
  }
}

When it happens

Trigger: A document missing source.ip, destination.ip, or any protocol hint (iana_number/network.transport), with the processor configured ignore_missing=false. buildFlow returns null at the first null IP or when protocol resolution yields null.

Common situations: Switching ignore_missing from true to false for stricter pipelines; events from sensors that emit only one side of a flow; documents where IP fields live under a different path than the configured source_ip/destination_ip.

Related errors

• field [{}] is null, cannot process it.
• invalid source port [{}]
• invalid destination port [{}]
• number [{}] must be a value between 0 and 65535
• unable to parse {} [{}]

AI-assisted analysis of elastic/elasticsearch@db6a809a66 (2026-08-12). Data as JSON: /api/errors/54d3c61b005cc9a5. Report an issue: GitHub.