{"record":{"id":"577271a9d489cc4b","repo":"risingwavelabs/risingwave","slug":"debezium-mongo-needs-a-id-column-in-table","errorCode":null,"errorMessage":"Debezium Mongo needs a `_id` column in table","messagePattern":"Debezium Mongo needs a `_id` column in table","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"src/connector/src/parser/debezium/mongo_json_parser.rs","lineNumber":84,"sourceCode":"            .context(\"Debezium Mongo needs a `_id` column with supported types (Varchar Jsonb int32 int64) in table\")?.clone();\n\n        if !props.strong_schema {\n            let _payload_column = rw_columns\n                .iter()\n                .find(|desc| desc.name == \"payload\" && matches!(desc.data_type, DataType::Jsonb))\n                .context(\n                    \"Debezium Mongo needs a `payload` column with supported types Jsonb in table\",\n                )?\n                .clone();\n\n            let columns = rw_columns\n                .iter()\n                .filter(|desc| desc.is_visible() && desc.additional_column.column_type.is_none())\n                .count();\n\n            // _rw_{connector}_file/partition & _rw_{connector}_offset are created automatically.\n            if columns != 2 || !rw_columns.iter().any(|desc| desc.name == \"_id\") {\n                bail!(\"Debezium Mongo needs a `_id` column in table\");\n            }\n        }\n\n        // encodings are fixed to MongoJson\n        let encoding = EncodingProperties::MongoJson(props);\n        // for key, it doesn't matter if strong schema is enabled or not\n        let key_builder = build_accessor_builder(encoding.clone())?;\n\n        let payload_builder = build_accessor_builder(encoding)?;\n\n        Ok(Self {\n            rw_columns,\n            source_ctx,\n            key_builder,\n            payload_builder,\n        })\n    }\n","sourceCodeStart":66,"sourceCodeEnd":102,"githubUrl":"https://github.com/risingwavelabs/risingwave/blob/6469eb736d691e8e9b8a419a57edd6429ca77417/src/connector/src/parser/debezium/mongo_json_parser.rs#L66-L102","documentation":"`DebeziumMongoJsonParser::new` validates that the target table schema is compatible with Debezium MongoDB CDC. Debezium emits MongoDB documents whose `_id` field is the key, so RisingWave requires exactly the `_id` column plus the two auto-generated offset/file columns (`_rw_{connector}_file` / `_rw_{connector}_offset`), i.e. exactly 2 visible non-additional columns including `_id`. Anything else is rejected at parser construction.","triggerScenarios":"Creating a Mongo CDC source/table whose visible column list does not contain a column named `_id`, or whose count of visible columns without additional-column types is not exactly 2 (the `_id` plus one offset column).","commonSituations":"Declaring a Mongo CDC table with renamed or extra business columns; forgetting the `_id` column in the CREATE TABLE; migrating a schema from a non-Mongo CDC connector where the `_id` requirement doesn't exist; auto schema derivation failing to include `_id`.","solutions":["Add an `_id` column to the source table definition (e.g. `_id JSONB PRIMARY KEY`).","Remove extra visible columns so only `_id` and the auto offset column remain; project extra fields downstream instead.","Confirm no additional_column types are attached that change the visible column count, and re-check `is_visible()` on each column desc."],"exampleFix":"-- before\nCREATE TABLE t (user_id JSONB, payload JSONB) WITH (\n  connector='mongodb-cdc', ...\n);\n-- after\nCREATE TABLE t (_id JSONB PRIMARY KEY, payload JSONB) WITH (\n  connector='mongodb-cdc', ...\n);","handlingStrategy":"validation","validationCode":"let visible: Vec<_> = rw_columns.iter().filter(|d| d.is_visible() && d.additional_column.column_type.is_none()).collect();\nassert!(visible.len() == 2 && visible.iter().any(|d| d.name == \"_id\"), \"table must define _id plus one offset column\");","typeGuard":null,"tryCatchPattern":"match res {\n    Err(e) if e.to_string().contains(\"needs a `_id` column\") => fix_table_schema_and_retry(),\n    Err(e) => return Err(e),\n    Ok(v) => v,\n}","preventionTips":["Always declare `_id JSONB PRIMARY KEY` in Mongo CDC table definitions.","Do not add extra visible columns to a Mongo CDC source; query them downstream instead.","Lint CREATE TABLE statements for Mongo CDC sources before applying."],"tags":["cdc","debezium","mongo","schema","primary-key"],"backgroundTag":"schema-validation-failed","analyzedSha":"6469eb736d691e8e9b8a419a57edd6429ca77417","analyzedAt":"2026-09-11T21:06:21.487Z","contentChangedAt":"2026-09-11T21:06:21.487Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}