{"record":{"id":"9b2f80b588887d6f","repo":"apache/beam","slug":"non-nullable-field-name-is-not-present-in-the-json-object","errorCode":null,"errorMessage":"Non-nullable field '{name}' is not present in the JSON object.","messagePattern":"Non-nullable field '(.+?)' is not present in the JSON object\\.","errorType":"validation","errorClass":"UnsupportedRowJsonException","httpStatus":null,"severity":"error","filePath":"sdks/java/core/src/main/java/org/apache/beam/sdk/util/RowJson.java","lineNumber":303,"sourceCode":"              throw new UnsupportedRowJsonException(\n                  \"Field '\" + fieldValue.name() + \"' is not present in the JSON object.\");\n          }\n        }\n\n        if (fieldValue.isJsonNull()) {\n          switch (this.nullBehavior) {\n            case ACCEPT_MISSING_OR_NULL:\n            case REQUIRE_NULL:\n              return null;\n            case REQUIRE_MISSING:\n              throw new UnsupportedRowJsonException(\n                  \"Field '\" + fieldValue.name() + \"' has a null value in the JSON object.\");\n          }\n        }\n      } else {\n        // field is not nullable\n        if (!fieldValue.isJsonValuePresent()) {\n          throw new UnsupportedRowJsonException(\n              \"Non-nullable field '\" + fieldValue.name() + \"' is not present in the JSON object.\");\n        } else if (fieldValue.isJsonNull()) {\n          throw new UnsupportedRowJsonException(\n              \"Non-nullable field '\" + fieldValue.name() + \"' has value null in the JSON object.\");\n        }\n      }\n\n      if (fieldValue.isRowType()) {\n        return jsonObjectToRow(fieldValue);\n      }\n\n      if (fieldValue.isArrayType()) {\n        return jsonArrayToList(fieldValue);\n      }\n\n      if (fieldValue.isMapType()) {\n        return jsonObjectToMap(fieldValue);\n      }","sourceCodeStart":285,"sourceCodeEnd":321,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/core/src/main/java/org/apache/beam/sdk/util/RowJson.java#L285-L321","documentation":"RowJson's Jackson deserializer builds Beam Rows from JSON. For a schema field declared non-nullable, the library requires the key to be present in the JSON object; if it is absent it throws UnsupportedRowJsonException. This enforces Beam Row's schema nullability contract at deserialization time.","triggerScenarios":"Calling RowJsonUtils.jsonToRow (directly or via ObjectMapper.readValue with a RowJson.RowJsonCreator deserializer) with a JSON object that omits a key for a non-nullable field of the Row schema.","commonSituations":"Producer schema evolved and a required field was dropped; hand-written JSON fixtures missing fields; upstream APIs omit keys instead of emitting nulls; user assumed absent keys default to a zero value.","solutions":["Add the missing field to the JSON input so every non-nullable schema field has a key","Make the schema field nullable via Schema.Field.nullable(...) if absence is acceptable","Pre-process/normalize the JSON to fill defaults for missing keys before parsing","Catch UnsupportedRowJsonException and skip/dead-letter the malformed record"],"exampleFix":"// before: schema has non-nullable field 'id'\n{\"name\": \"x\"}\n// after\n{\"id\": \"abc\", \"name\": \"x\"}","handlingStrategy":"validation","validationCode":"Set<String> present = json.fields().hasNext() ? new HashSet<>() : new HashSet<>();\njson.fieldNames().forEachRemaining(present::add);\nList<String> missing = schema.getFields().stream()\n    .filter(f -> !f.getType().getNullable() && !present.contains(f.getName()))\n    .map(Schema.Field::getName).collect(Collectors.toList());\nif (!missing.isEmpty()) throw new IllegalArgumentException(\"Missing non-nullable fields: \" + missing);","typeGuard":null,"tryCatchPattern":"try { Row r = RowJsonUtils.jsonToRow(mapper, json); } catch (RowJson.UnsupportedRowJsonException e) { /* dead-letter e */ }","preventionTips":["Validate payloads against the schema before parsing","Default-fill missing keys in an ingestion preprocessor","Keep producer and reader schemas in sync (schema registry)"],"tags":["beam","json","schema","deserialization"],"backgroundTag":"schema-validation-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}