{"record":{"id":"62436c08ef0b868d","repo":"elastic/elasticsearch","slug":"field-is-null-cannot-process-it-62436c","errorCode":null,"errorMessage":"field [{}] is null, cannot process it.","messagePattern":"field \\[(.+?)\\] is null, cannot process it\\.","errorType":"exception","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"modules/ingest-common/src/main/java/org/elasticsearch/ingest/common/CsvProcessor.java","lineNumber":80,"sourceCode":"        this.headers = headers;\n        this.trim = trim;\n        this.quote = quote;\n        this.separator = separator;\n        this.ignoreMissing = ignoreMissing;\n        this.emptyValue = emptyValue;\n    }\n\n    @Override\n    public IngestDocument execute(IngestDocument ingestDocument) {\n        if (headers.length == 0) {\n            return ingestDocument;\n        }\n\n        String line = ingestDocument.getFieldValue(field, String.class, ignoreMissing);\n        if (line == null && ignoreMissing) {\n            return ingestDocument;\n        } else if (line == null) {\n            throw new IllegalArgumentException(\"field [\" + field + \"] is null, cannot process it.\");\n        }\n        new CsvParser(ingestDocument, quote, separator, trim, headers, emptyValue).process(line);\n        return ingestDocument;\n    }\n\n    @Override\n    public String getType() {\n        return TYPE;\n    }\n\n    public static final class Factory implements org.elasticsearch.ingest.Processor.Factory {\n        @Override\n        public CsvProcessor create(\n            Map<String, Processor.Factory> registry,\n            String processorTag,\n            String description,\n            Map<String, Object> config,\n            ProjectId projectId","sourceCodeStart":62,"sourceCodeEnd":98,"githubUrl":"https://github.com/elastic/elasticsearch/blob/db6a809a667c081ca1dc7500389d26975573215f/modules/ingest-common/src/main/java/org/elasticsearch/ingest/common/CsvProcessor.java#L62-L98","documentation":"Thrown by CsvProcessor.execute when the configured source field is null or absent and 'ignore_missing' is false (default). Identical pattern to other ingest processors: getFieldValue is called with ignoreMissing, and a null result without ignore_missing short-circuits is an unrecoverable error. Note 'headers' being empty causes an early no-op return before this check.","triggerScenarios":"A csv processor configured on a field that is null or absent in the incoming document, with 'ignore_missing' omitted or set to false.","commonSituations":"Optional message fields that are sometimes absent; pipeline reused across multiple ingest streams where only some have the CSV field; field renamed upstream and the csv processor still references the old name.","solutions":["Set 'ignore_missing: true' on the csv processor to skip documents lacking the field.","Verify the configured 'field' name matches what prior processors (or the source) produce.","Add a default value via a 'set' processor conditioned on field presence before the csv step."],"exampleFix":"// before\n{\"csv\": {\"field\": \"message\", \"target_fields\": [\"a\",\"b\"]}}\n// after\n{\"csv\": {\"field\": \"message\", \"target_fields\": [\"a\",\"b\"], \"ignore_missing\": true}}","handlingStrategy":"validation","validationCode":"// Default ignore_missing on the csv processor for optional message fields.\n{\"csv\": {\"field\": \"message\", \"target_fields\": [\"a\",\"b\"], \"ignore_missing\": true}}","typeGuard":null,"tryCatchPattern":"{\"on_failure\": [{\"index\": {\"index\": \"ingest-dlq\"}}]}","preventionTips":["Set ignore_missing: true when the field is optional.","Verify field names after any upstream rename.","Use a set processor to default empty values if absence should be normalized."],"tags":["ingest","csv-processor","null-handling","config"],"analyzedSha":"db6a809a667c081ca1dc7500389d26975573215f","analyzedAt":"2026-08-12T01:39:14.192Z","schemaVersion":2},"datasetVersion":"2026-08-12T12:17:08.281Z"}