{"record":{"id":"5ed7c7bc82e6aa8f","repo":"apache/flink","slug":"some-field-is-missing-in-the-json-data-5ed7c7","errorCode":null,"errorMessage":"Some field is missing in the Json data.","messagePattern":"Some field is missing in the Json data\\.","errorType":"exception","errorClass":"org.apache.flink.formats.json.JsonParseException","httpStatus":null,"severity":"error","filePath":"flink-formats/flink-json/src/main/java/org/apache/flink/formats/json/JsonParserToRowDataConverters.java","lineNumber":630,"sourceCode":"            jp.nextToken();\n            while (jp.currentToken() != JsonToken.END_OBJECT) {\n                if (cnt >= arity) {\n                    skipToNextField(jp);\n                    continue;\n                }\n                String fieldName = jp.getText();\n                jp.nextToken();\n                ProjectedConverter converter = fieldConverters.get(fieldName);\n                if (converter != null) {\n                    converter.convert(jp, outputRow);\n                    jp.nextToken();\n                    cnt++;\n                } else {\n                    skipToNextField(jp);\n                }\n            }\n            if (cnt < arity && failOnMissingField) {\n                throw new JsonParseException(\"Some field is missing in the Json data.\");\n            }\n        }\n    }\n}\n","sourceCodeStart":612,"sourceCodeEnd":635,"githubUrl":"https://github.com/apache/flink/blob/2f3c205e9266cb30240eb7f4fdab15cad629a70f/flink-formats/flink-json/src/main/java/org/apache/flink/formats/json/JsonParserToRowDataConverters.java#L612-L635","documentation":"JsonParseException from RowNestedConverter.convertNotNull: the nested-projection variant of the missing-field check. After parsing the object, fewer projected fields were found than expected while 'json.fail-on-missing-field'='true'. It applies to the nested fields selected via projection, not the full row.","triggerScenarios":"failOnMissingField=true, a projection over nested ROW fields, and a record where one of those nested keys is absent from its JSON object.","commonSituations":"Optional nested attributes modeled as required; producers occasionally omitting empty sub-objects; enabling fail-on-missing-field while migrating schemas where nested keys come and go.","solutions":["Set 'json.fail-on-missing-field'='false' so absent nested fields deserialize as null","Have the producer always emit the nested keys (explicit nulls)","Restrict projections to nested fields guaranteed present, or handle nulls downstream"],"exampleFix":"-- before\n'json.fail-on-missing-field' = 'true'\n\n-- after\n'json.fail-on-missing-field' = 'false'","handlingStrategy":"validation","validationCode":"// before enabling fail-on-missing-field with nested projections:\n// verify every projected nested key exists in representative records\nfor (String sample : samples) {\n    JsonNode root = mapper.readTree(sample);\n    for (String path : projectedNestedPaths) {\n        Preconditions.checkState(resolve(root, path) != null, \"missing \" + path);\n    }\n}","typeGuard":null,"tryCatchPattern":"catch (JsonParseException e) {\n    if (e.getMessage().contains(\"field is missing\")) {\n        // absent nested key with fail-on-missing-field=true; disable option or fix producer\n    }\n}","preventionTips":["Nested optional fields + fail-on-missing-field is the classic trigger; pick one","Have producers emit null-valued nested keys instead of omitting them"],"tags":["flink","json","missing-field","nested-projection","options"],"backgroundTag":null,"analyzedSha":"2f3c205e9266cb30240eb7f4fdab15cad629a70f","analyzedAt":"2026-08-14T08:48:24.518Z","schemaVersion":2},"datasetVersion":"2026-08-14T10:17:34.591Z"}