{"record":{"id":"54d3c61b005cc9a5","repo":"elastic/elasticsearch","slug":"unable-to-construct-flow-from-document","errorCode":null,"errorMessage":"unable to construct flow from document","messagePattern":"unable to construct flow from document","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"modules/ingest-common/src/main/java/org/elasticsearch/ingest/common/CommunityIdProcessor.java","lineNumber":148,"sourceCode":"        return ignoreMissing;\n    }\n\n    @Override\n    public IngestDocument execute(IngestDocument document) throws Exception {\n        String sourceIp = document.getFieldValue(sourceIpField, String.class, ignoreMissing);\n        String destinationIp = document.getFieldValue(destinationIpField, String.class, ignoreMissing);\n        Object ianaNumber = document.getFieldValue(ianaNumberField, Object.class, true);\n        Supplier<Object> transport = () -> document.getFieldValue(transportField, Object.class, ignoreMissing);\n        Supplier<Object> sourcePort = () -> document.getFieldValue(sourcePortField, Object.class, ignoreMissing);\n        Supplier<Object> destinationPort = () -> document.getFieldValue(destinationPortField, Object.class, ignoreMissing);\n        Object icmpType = document.getFieldValue(icmpTypeField, Object.class, true);\n        Object icmpCode = document.getFieldValue(icmpCodeField, Object.class, true);\n        Flow flow = buildFlow(sourceIp, destinationIp, ianaNumber, transport, sourcePort, destinationPort, icmpType, icmpCode);\n        if (flow == null) {\n            if (ignoreMissing) {\n                return document;\n            } else {\n                throw new IllegalArgumentException(\"unable to construct flow from document\");\n            }\n        }\n\n        document.setFieldValue(targetField, flow.toCommunityId(seed));\n        return document;\n    }\n\n    public static String apply(\n        String sourceIpAddrString,\n        String destIpAddrString,\n        Object ianaNumber,\n        Object transport,\n        Object sourcePort,\n        Object destinationPort,\n        Object icmpType,\n        Object icmpCode,\n        int seed\n    ) {","sourceCodeStart":130,"sourceCodeEnd":166,"githubUrl":"https://github.com/elastic/elasticsearch/blob/db6a809a667c081ca1dc7500389d26975573215f/modules/ingest-common/src/main/java/org/elasticsearch/ingest/common/CommunityIdProcessor.java#L130-L166","documentation":"CommunityIdProcessor.execute calls buildFlow, which returns null when source.ip, destination.ip, or both are absent, or when both iana_number and transport are missing. If ignore_missing is false (note: the default for CommunityIdProcessor is true, so this throw is opt-in), execute throws this message instead of passing the document through.","triggerScenarios":"A document missing source.ip, destination.ip, or any protocol hint (iana_number/network.transport), with the processor configured ignore_missing=false. buildFlow returns null at the first null IP or when protocol resolution yields null.","commonSituations":"Switching ignore_missing from true to false for stricter pipelines; events from sensors that emit only one side of a flow; documents where IP fields live under a different path than the configured source_ip/destination_ip.","solutions":["Confirm source.ip and destination.ip (or your configured field paths) are populated on every document before community_id runs.","If missing values are legitimate, leave ignore_missing at its default (true) — the processor will pass such documents through unchanged.","Add a mapping/ enrichment step that backfills network.iana_number or network.transport so buildFlow can resolve the protocol.","Pre-route documents lacking the required fields away from the community_id processor."],"exampleFix":"// before — strict config, event with no IPs fails\n//   { \"community_id\": { \"ignore_missing\": false } }\n//\n// after — tolerate events that lack a complete flow\n//   { \"community_id\": { \"ignore_missing\": true } }","handlingStrategy":"validation","validationCode":"// All three are required to build a flow.\nboolean canBuildFlow(IngestDocument d) {\n    return d.hasField(\"source.ip\") && d.hasField(\"destination.ip\")\n        && (d.hasField(\"network.iana_number\") || d.hasField(\"network.transport\"));\n}","typeGuard":null,"tryCatchPattern":"{\n  \"community_id\": {\n    \"ignore_missing\": true,\n    \"on_failure\": [\n      { \"set\": { \"field\": \"ingest.error\", \"value\": \"community-id-missing-flow\" } },\n      { \"redirect\": { \"pipeline\": \"quarantine\" } }\n    ]\n  }\n}","preventionTips":["Keep ignore_missing at its default (true) for community_id in heterogeneous pipelines.","Ensure source.ip, destination.ip, and at least one of iana_number/transport are populated before community_id.","Verify the configured field paths match your document schema — defaults are ECS."],"tags":["ingest","community-id","missing-field","network-flow"],"analyzedSha":"db6a809a667c081ca1dc7500389d26975573215f","analyzedAt":"2026-08-12T01:39:14.192Z","schemaVersion":2},"datasetVersion":"2026-08-12T08:17:17.861Z"}