{"record":{"id":"2e23db0fb2b7939c","repo":"apache/iceberg","slug":"failed-to-find-field-key-in-schema-schema-n","errorCode":null,"errorMessage":"Failed to find field '${key}' in schema ${schema.name()}","messagePattern":"Failed to find field '(.+?)' in schema (.+?)","errorType":"exception","errorClass":"DataException","httpStatus":null,"severity":"error","filePath":"kafka-connect/kafka-connect-transforms/src/main/java/org/debezium/connector/mongodb/transforms/MongoDataConverter.java","lineNumber":143,"sourceCode":"      case REGULAR_EXPRESSION:\n        Struct regexStruct = new Struct(schema.field(key).schema());\n        regexStruct.put(\"regex\", keyValueForStruct.getValue().asRegularExpression().getPattern());\n        regexStruct.put(\"options\", keyValueForStruct.getValue().asRegularExpression().getOptions());\n        colValue = regexStruct;\n        break;\n\n      case TIMESTAMP:\n        colValue = new Date(1000L * keyValueForStruct.getValue().asTimestamp().getTime());\n        break;\n\n      case DECIMAL128:\n        colValue = keyValueForStruct.getValue().asDecimal128().getValue().toString();\n        break;\n\n      case DOCUMENT:\n        Field field = schema.field(key);\n        if (field == null) {\n          throw new DataException(\"Failed to find field '\" + key + \"' in schema \" + schema.name());\n        }\n        Schema documentSchema = field.schema();\n        Struct documentStruct = new Struct(documentSchema);\n        BsonDocument docs = keyValueForStruct.getValue().asDocument();\n\n        for (Entry<String, BsonValue> doc : docs.entrySet()) {\n          convertFieldValue(doc, documentStruct, documentSchema);\n        }\n\n        colValue = documentStruct;\n        break;\n\n      case ARRAY:\n        if (keyValueForStruct.getValue().asArray().isEmpty()) {\n          switch (arrayEncoding) {\n            case ARRAY:\n              colValue = Lists.newArrayList();\n              break;","sourceCodeStart":125,"sourceCodeEnd":161,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/kafka-connect/kafka-connect-transforms/src/main/java/org/debezium/connector/mongodb/transforms/MongoDataConverter.java#L125-L161","documentation":"MongoDataConverter.convertFieldValue throws DataException when converting a BSON DOCUMENT field: the matching Field cannot be found by key in the target Kafka Connect struct schema. The schema was built from one document but a record contains a document with fields absent from that schema.","triggerScenarios":"Calling convertRecord/convertFieldValue on a record whose BSON document contains a sub-document key that is not present in the pre-built Connect schema for that struct (schema.name() lookup returns null).","commonSituations":"MongoDB collections with heterogeneous documents — new fields added to some documents after the schema was inferred from the first sample; schema drift between documents in the same collection.","solutions":["Ensure all documents in the collection share the same field set, or remove/null out inconsistent sub-documents.","Re-run schema inference so the Connect schema includes the new field.","Pre-create a schema covering all expected document keys."],"exampleFix":null,"handlingStrategy":"type-guard","validationCode":"// Before converting, verify every document key exists in the schema\nBsonDocument doc = ...;\nfor (String k : doc.keySet()) {\n  if (schema.field(k) == null) throw new IllegalStateException(\"document key missing from schema: \" + k);\n}","typeGuard":"boolean schemaCovers(org.apache.kafka.connect.data.Schema schema, BsonDocument doc) { return doc.keySet().stream().allMatch(k -> schema.field(k) != null); }","tryCatchPattern":"try { ... } catch (org.apache.kafka.connect.errors.DataException e) { log.error(\"Schema drift detected: {}\", e.getMessage()); /* re-infer schema or DLQ */ }","preventionTips":["Keep MongoDB documents homogeneous within a collection","Version schema changes and re-run schema inference before migrating new fields","Monitor for new fields with a schema validation rule in MongoDB"],"tags":["kafka-connect","debezium","mongodb","schema"],"backgroundTag":"schema-validation-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}