{"record":{"id":"8780ad5ebc7e35a4","repo":"risingwavelabs/risingwave","slug":"field-not-found-in-our-schema","errorCode":null,"errorMessage":"Field {} not found in our schema","messagePattern":"Field (.+?) not found in our schema","errorType":"exception","errorClass":"anyhow::Error","httpStatus":null,"severity":"error","filePath":"src/connector/src/sink/iceberg/create_table.rs","lineNumber":377,"sourceCode":"                }\n                if !field_is_compatible(rw_type, arrow_field)? {\n                    return Ok(false);\n                }\n            }\n            Ok(true)\n        }\n        (_, left, right) => Ok(left.equals_datatype(right)),\n    }\n}\n\nfn check_compatibility(\n    schema_fields: HashMap<&str, &risingwave_common::types::DataType>,\n    fields: &ArrowFields,\n) -> anyhow::Result<bool> {\n    for arrow_field in fields {\n        let our_field_type = schema_fields\n            .get(arrow_field.name().as_str())\n            .ok_or_else(|| anyhow!(\"Field {} not found in our schema\", arrow_field.name()))?;\n\n        if !field_is_compatible(our_field_type, arrow_field)? {\n            let converted_arrow_data_type = IcebergArrowConvert\n                .to_arrow_field(\"\", our_field_type)\n                .map_err(|e| anyhow!(e))?\n                .data_type()\n                .clone();\n            bail!(\n                \"field {}'s type is incompatible\\nRisingWave converted data type: {}\\niceberg's data type: {}\",\n                arrow_field.name(),\n                converted_arrow_data_type,\n                arrow_field.data_type()\n            );\n        }\n    }\n    Ok(true)\n}\n","sourceCodeStart":359,"sourceCodeEnd":395,"githubUrl":"https://github.com/risingwavelabs/risingwave/blob/6469eb736d691e8e9b8a419a57edd6429ca77417/src/connector/src/sink/iceberg/create_table.rs#L359-L395","documentation":"During sink-vs-Iceberg-table compatibility validation, `check_compatibility` looks up each Iceberg arrow field's name in the RisingWave schema field map. If the Iceberg table contains a column that does not exist in the RisingWave sink schema, it throws `Field {name} not found in our schema`. The two schemas must reference exactly the same column set.","triggerScenarios":"Called from `try_matches_arrow_schema` when validating an existing Iceberg table's arrow schema against the sink's RW schema: the Iceberg table has a column name absent from the RW schema (extra column in the table, or the sink selects a subset of columns).","commonSituations":"Iceberg table was created/altered externally with extra columns; sink SQL selects fewer/differently-named columns than the table; schema evolution added a column after sink creation; case-sensitivity differences between catalogs.","solutions":["Align the sink query with the Iceberg table: SELECT exactly the same columns (names and set) as the table has.","If the table has evolved, alter the sink (recreate) to include the new columns in matching order.","Check for case mismatches (e.g., table column `Id` vs sink `id`) and quote/normalize names."],"exampleFix":"// before: table has (a, b, c), sink only selects (a, b)\nCREATE SINK s AS SELECT a, b FROM t;\n// after\nCREATE SINK s AS SELECT a, b, c FROM t;","handlingStrategy":"validation","validationCode":"// verify sink columns equal the Iceberg table columns before creating\nlet sink_cols: HashSet<&str> = rw_schema.fields.iter().map(|f| f.name.as_str()).collect();\nlet table_cols: HashSet<&str> = arrow_schema.fields().iter().map(|f| f.name()).collect();\nif sink_cols != table_cols {\n    return Err(format!(\"column set mismatch: sink-only={:?} table-only={:?}\",\n        sink_cols - table_cols, table_cols - sink_cols));\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Always generate the sink column list from the Iceberg table's current schema.","Re-create sinks after Iceberg table schema evolution.","Use consistent identifier casing between RW and the catalog."],"tags":["iceberg","schema","column-missing","compatibility"],"backgroundTag":"schema-validation-failed","analyzedSha":"6469eb736d691e8e9b8a419a57edd6429ca77417","analyzedAt":"2026-09-11T21:06:21.487Z","contentChangedAt":"2026-09-11T21:06:21.487Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}