{"record":{"id":"b0ff80d940c8a19a","repo":"apache/flink","slug":"some-field-is-missing-in-the-json-data","errorCode":null,"errorMessage":"Some field is missing in the JSON data.","messagePattern":"Some field is missing in the JSON data\\.","errorType":"exception","errorClass":"org.apache.flink.formats.json.JsonParseException","httpStatus":null,"severity":"error","filePath":"flink-formats/flink-json/src/main/java/org/apache/flink/formats/json/JsonParserToRowDataConverters.java","lineNumber":436,"sourceCode":"                String fieldName = jp.getText();\n                jp.nextToken();\n                Integer idx = nameIdxMap.get(fieldName);\n                if (idx != null) {\n                    try {\n                        Object convertField = fieldConverters[idx].convert(jp);\n                        row.setField(idx, convertField);\n                    } catch (Throwable t) {\n                        throw new JsonParseException(\n                                String.format(\"Fail to deserialize at field: %s.\", fieldName));\n                    }\n                    jp.nextToken();\n                    cnt++;\n                } else {\n                    skipToNextField(jp);\n                }\n            }\n            if (cnt < arity && failOnMissingField) {\n                throw new JsonParseException(\"Some field is missing in the JSON data.\");\n            }\n            return row;\n        };\n    }\n\n    public static void skipToNextField(JsonParser jp) throws IOException {\n        switch (jp.currentToken()) {\n            case START_OBJECT:\n            case START_ARRAY:\n                int match = 1;\n                JsonToken token;\n                while (match > 0) {\n                    token = jp.nextToken();\n                    if (token == JsonToken.END_ARRAY || token == JsonToken.END_OBJECT) {\n                        match--;\n                    } else if (token == JsonToken.START_ARRAY || token == JsonToken.START_OBJECT) {\n                        match++;\n                    }","sourceCodeStart":418,"sourceCodeEnd":454,"githubUrl":"https://github.com/apache/flink/blob/2f3c205e9266cb30240eb7f4fdab15cad629a70f/flink-formats/flink-json/src/main/java/org/apache/flink/formats/json/JsonParserToRowDataConverters.java#L418-L454","documentation":"JsonParseException from createRowConverter after the object is fully parsed: fewer known fields were present than the row arity while 'json.fail-on-missing-field'='true'. It enforces the strict policy you opted into: every declared column must appear in the JSON object (extra unknown fields are skipped, missing ones are fatal).","triggerScenarios":"failOnMissingField=true and a record omits one or more declared columns (absent key, not null-valued — explicit nulls count as present).","commonSituations":"Optional fields modeled as required columns; upstream schema evolution dropping fields; turning on the flag for tolerance of nulls but expecting absent keys to be null too.","solutions":["Set 'json.fail-on-missing-field'='false' (absent fields become null) if absence is legitimate","Make the producer always emit all declared fields (explicit nulls)","Keep the flag true only when the contract really requires every field"],"exampleFix":"-- before\n'json.fail-on-missing-field' = 'true'\n\n-- after\n'json.fail-on-missing-field' = 'false'","handlingStrategy":"validation","validationCode":"// policy check before enabling: can every declared column be absent in real data?\n// if yes -> do not enable fail-on-missing-field\nif (optionalFieldsExist(rowType)) {\n    Preconditions.checkArgument(!failOnMissingField, \"optional columns present; disable fail-on-missing-field\");\n}","typeGuard":null,"tryCatchPattern":"catch (JsonParseException e) {\n    if (e.getMessage().contains(\"field is missing\")) {\n        // either fix producer to emit all fields, or disable the option\n    }\n}","preventionTips":["Only enable fail-on-missing-field when the producer contract guarantees all keys","Prefer explicit nulls from producers over absent keys for nullable columns"],"tags":["flink","json","missing-field","options","schema"],"backgroundTag":null,"analyzedSha":"2f3c205e9266cb30240eb7f4fdab15cad629a70f","analyzedAt":"2026-08-14T08:48:24.518Z","schemaVersion":2},"datasetVersion":"2026-08-14T10:17:34.591Z"}