{"record":{"id":"494a109b2a8f0b82","repo":"risingwavelabs/risingwave","slug":"error-from-iceberg-library-err-equalitydeletew","errorCode":null,"errorMessage":"error from iceberg library: {err} (EqualityDeleteWriterConfig::new failed)","messagePattern":"error from iceberg library: (.+?) \\(EqualityDeleteWriterConfig::new failed\\)","errorType":"exception","errorClass":"SinkError::Iceberg","httpStatus":null,"severity":"error","filePath":"src/connector/src/sink/iceberg/writer.rs","lineNumber":578,"sourceCode":"                table.file_io().clone(),\n                DefaultLocationGenerator::new(table.metadata())\n                    .map_err(|err| SinkError::Iceberg(anyhow!(err)))?,\n                DefaultFileNameGenerator::new(\n                    writer_param.actor_id.to_string(),\n                    Some(format!(\"pos-del-{}\", unique_uuid_suffix)),\n                    iceberg::spec::DataFileFormat::Parquet,\n                ),\n            );\n            PositionDeleteWriterBuilderType::PositionDelete(PositionDeleteFileWriterBuilder::new(\n                rolling_writer_builder,\n            ))\n        };\n        let equality_delete_builder = {\n            let eq_del_config = EqualityDeleteWriterConfig::new(\n                unique_column_ids.clone(),\n                table.metadata().current_schema().clone(),\n            )\n            .map_err(|err| SinkError::Iceberg(anyhow!(err)))?;\n            let parquet_writer_builder = ParquetWriterBuilder::new(\n                parquet_writer_properties,\n                Arc::new(\n                    arrow_schema_to_schema(eq_del_config.projected_arrow_schema_ref())\n                        .map_err(|err| SinkError::Iceberg(anyhow!(err)))?,\n                ),\n            );\n            let rolling_writer_builder = RollingFileWriterBuilder::new(\n                parquet_writer_builder,\n                (config.target_file_size_mb() * 1024 * 1024) as usize,\n                table.file_io().clone(),\n                DefaultLocationGenerator::new(table.metadata())\n                    .map_err(|err| SinkError::Iceberg(anyhow!(err)))?,\n                DefaultFileNameGenerator::new(\n                    writer_param.actor_id.to_string(),\n                    Some(format!(\"eq-del-{}\", unique_uuid_suffix)),\n                    iceberg::spec::DataFileFormat::Parquet,\n                ),","sourceCodeStart":560,"sourceCodeEnd":596,"githubUrl":"https://github.com/risingwavelabs/risingwave/blob/6469eb736d691e8e9b8a419a57edd6429ca77417/src/connector/src/sink/iceberg/writer.rs#L560-L596","documentation":"EqualityDeleteWriterConfig::new(unique_column_ids, current_schema) failed in iceberg-rs while setting up the equality-delete writer. This constructor validates that the given column ids exist in the schema and are usable as an equality-delete key set; any mismatch aborts build_upsert with SinkError::Iceberg.","triggerScenarios":"Calling build_upsert when unique_column_ids (derived from the sink's primary key via resolve_equality_delete_field_ids) contains an id not present in table.metadata().current_schema(), an empty id set, or a projected schema inconsistent with the ids.","commonSituations":"Iceberg schema evolution (ids dropped/changed) after the sink computed its column ids; stale/mismatched sink configuration pointing at a table whose key columns differ; empty PK mapping producing an empty id list rejected by the library.","solutions":["Re-derive unique_column_ids from the current table schema via iceberg_schema.field_id_by_name for each PK column and confirm none are None","Verify the sink's PRIMARY KEY columns still exist in the Iceberg table (schema evolution may have removed them) and update the sink or table accordingly","Read the wrapped iceberg library error to see which specific column id failed validation","Ensure unique_column_ids is non-empty before constructing EqualityDeleteWriterConfig"],"exampleFix":"// before\nlet eq_del_config = EqualityDeleteWriterConfig::new(\n    unique_column_ids.clone(),\n    table.metadata().current_schema().clone(),\n)\n.map_err(|err| SinkError::Iceberg(anyhow!(err)))?;\n// after\nanyhow::ensure!(!unique_column_ids.is_empty(), \"equality delete needs >= 1 column id\");\nlet eq_del_config = EqualityDeleteWriterConfig::new(\n    unique_column_ids.clone(),\n    table.metadata().current_schema().clone(),\n)\n.map_err(|err| {\n    SinkError::Iceberg(anyhow::anyhow!(\n        \"EqualityDeleteWriterConfig::new failed: {err}; ids = {:?}\",\n        unique_column_ids\n    ))\n})?","handlingStrategy":"validation","validationCode":"anyhow::ensure!(!unique_column_ids.is_empty(), \"equality delete config requires at least one column id\");\nlet schema_field_ids: std::collections::HashSet<i32> =\n    table.metadata().current_schema().fields().map(|f| f.id).collect();\nanyhow::ensure!(\n    unique_column_ids.iter().all(|id| schema_field_ids.contains(id)),\n    \"equality delete ids not present in schema\"\n);","typeGuard":null,"tryCatchPattern":"let eq_del_config = EqualityDeleteWriterConfig::new(\n    unique_column_ids.clone(),\n    table.metadata().current_schema().clone(),\n)\n.map_err(|err| SinkError::Iceberg(anyhow!(\"EqualityDeleteWriterConfig::new failed: {err} (ids: {:?})\", unique_column_ids)))?;","preventionTips":["Re-resolve PK column ids from the current table schema at writer build time, never cache across schema evolutions","Confirm all PK columns still exist after any Iceberg ALTER/DROP COLUMN","Ensure the PK set is non-empty before building the equality-delete writer","Include the offending column ids in the wrapped error message"],"tags":["rust","iceberg","equality-delete","primary-key"],"backgroundTag":"invalid-config-value","analyzedSha":"6469eb736d691e8e9b8a419a57edd6429ca77417","analyzedAt":"2026-09-11T21:06:21.487Z","contentChangedAt":"2026-09-11T21:06:21.487Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}