{"record":{"id":"b9157ad4cfe991fc","repo":"apache/beam","slug":"input-schema-must-contain-document-id-field-documentidfield","errorCode":null,"errorMessage":"Input schema must contain document id field: {documentIdField}","messagePattern":"Input schema must contain document id field: (.+?)","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/firestore/FirestoreWriteSchemaTransformProvider.java","lineNumber":102,"sourceCode":"    private final FirestoreWriteSchemaTransformConfiguration configuration;\n\n    FirestoreWriteSchemaTransform(FirestoreWriteSchemaTransformConfiguration configuration) {\n      configuration.validate();\n      this.configuration = configuration;\n    }\n\n    @Override\n    public PCollectionRowTuple expand(PCollectionRowTuple input) {\n      PCollection<Row> rows = input.get(INPUT_TAG);\n      Schema inputSchema = rows.getSchema();\n      String projectId = resolveProjectId(input.getPipeline());\n      String databaseId = resolveDatabaseId(input.getPipeline());\n      String documentIdField =\n          Strings.isNullOrEmpty(configuration.getDocumentIdField())\n              ? DEFAULT_DOCUMENT_ID_FIELD\n              : configuration.getDocumentIdField();\n      if (!inputSchema.hasField(documentIdField)) {\n        throw new IllegalArgumentException(\n            \"Input schema must contain document id field: \" + documentIdField);\n      }\n\n      boolean handleErrors = ErrorHandling.hasOutput(configuration.getErrorHandling());\n      Schema errorSchema = ErrorHandling.errorSchema(inputSchema);\n\n      PCollectionTuple outputTuple =\n          rows.apply(\n              \"ConvertToFirestoreWrite\",\n              ParDo.of(\n                      new RowToWriteFn(\n                          inputSchema,\n                          projectId,\n                          databaseId,\n                          configuration.getCollectionId(),\n                          documentIdField,\n                          handleErrors,\n                          errorSchema))","sourceCodeStart":84,"sourceCodeEnd":120,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/firestore/FirestoreWriteSchemaTransformProvider.java#L84-L120","documentation":"When expanding the Firestore write SchemaTransform, the input schema must contain the field used as the Firestore document id (defaults to a configured documentIdField, or a default like '__id__'). The configured field is absent from the input PCollection's schema, so the transform fails fast with this IllegalArgumentException during pipeline construction.","triggerScenarios":"FirestoreIO.write()/FirestoreWriteSchemaTransformProvider configured with documentIdField=\"X\" where the incoming table's Schema has no field named X; or using the default document id field while the input schema lacks it.","commonSituations":"Renaming the id column upstream without updating configuration; feeding a schema-transform table whose column names differ (case sensitivity); using the default id field name with custom input schemas.","solutions":["Add the configured document id field to the input schema, or change documentIdField to an existing field name","Check exact case and spelling of the field name — schema field lookup is exact","Generate the id in a prior step (e.g. withConstantFields or a MapElements adding an id column)","If relying on the default, either name your id column the default name or explicitly set documentIdField"],"exampleFix":"// before\n.write().to(\"books\").withDocumentIdField(\"bookId\") // schema has only 'id'\n// after\n.write().to(\"books\").withDocumentIdField(\"id\") // or rename the column to bookId","handlingStrategy":"validation","validationCode":"if (!input.getSchema().hasField(documentIdField)) { throw new IllegalArgumentException(\"schema missing id field: \" + documentIdField); }","typeGuard":null,"tryCatchPattern":"try { write.expand(input); } catch (IllegalArgumentException e) { if (e.getMessage().contains(\"Input schema must contain document id field\")) { // fix schema or configuration then rebuild } throw e; }","preventionTips":["Validate that documentIdField names an existing column before expanding the transform","Keep id-column naming consistent between producers and Firestore config","Print input.getSchema() when debugging schema-transform mismatches"],"tags":["firestore","beam","schema","configuration","pipeline-configuration"],"backgroundTag":"schema-validation-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}