{"record":{"id":"6bef837a136085bc","repo":"risingwavelabs/risingwave","slug":"delete-field-id-not-found-in-schema","errorCode":null,"errorMessage":"Delete field id {} not found in schema","messagePattern":"Delete field id (.+?) not found in schema","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"src/connector/src/source/iceberg/mod.rs","lineNumber":422,"sourceCode":"                    iceberg::spec::DataContentType::PositionDeletes => {\n                        if position_delete_files_set.insert(delete_file.file_path.clone()) {\n                            position_delete_files.push(delete_file.to_file_scan_task(&task));\n                        }\n                    }\n                }\n            }\n\n            // Top-level scan tasks always represent data files. Keep their delete\n            // descriptors intact so the SDK reader can apply them when requested.\n            data_files.push(task);\n        }\n        let schema = table_schema.clone();\n        let equality_delete_columns = equality_delete_ids\n            .unwrap_or_default()\n            .into_iter()\n            .map(|id| match schema.name_by_field_id(id) {\n                Some(name) => Ok::<std::string::String, ConnectorError>(name.to_owned()),\n                None => bail!(\"Delete field id {} not found in schema\", id),\n            })\n            .collect::<ConnectorResult<Vec<_>>>()?;\n\n        Ok(IcebergListResult {\n            data_files,\n            equality_delete_files,\n            position_delete_files,\n            equality_delete_columns,\n            format_version,\n            schema,\n        })\n    }\n\n    /// Uniformly distribute scan tasks to compute nodes.\n    /// It's deterministic so that it can best utilize the data locality.\n    ///\n    /// # Arguments\n    /// * `file_scan_tasks`: The file scan tasks to be split.","sourceCodeStart":404,"sourceCodeEnd":440,"githubUrl":"https://github.com/risingwavelabs/risingwave/blob/6469eb736d691e8e9b8a419a57edd6429ca77417/src/connector/src/source/iceberg/mod.rs#L404-L440","documentation":"After collecting equality delete files, the planner resolves each equality delete field id to a column name using the table schema (`schema.name_by_field_id`). If a delete field id has no corresponding field in the current schema, the id cannot be projected and the function bails with this error naming the missing id.","triggerScenarios":"`list_scan_tasks` resolving `equality_delete_ids` where one of the ids was removed from the Iceberg schema (dropped column) or comes from a delete file written against an incompatible schema version.","commonSituations":"Iceberg schema evolution that dropped a column previously used in equality deletes; delete files from an old schema applied to the current schema; mismatch between table schema loaded and the schema the deletes were written with.","solutions":["Restore/re-add the missing field to the Iceberg schema so the field id resolves again.","Compact/rewrite delete files so stale deletes referencing dropped fields are removed.","Avoid dropping columns that are used as equality-delete keys, or first rewrite the affected files.","Verify the source loads the correct table/schema (right table identifier and catalog)."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"let missing: Vec<i64> = equality_ids.iter()\n    .filter(|id| schema.name_by_field_id(**id).is_none())\n    .copied().collect();\nif !missing.is_empty() { /* heal schema or rewrite deletes before scanning */ }","typeGuard":"fn ids_resolvable(schema: &Schema, ids: &[i64]) -> bool {\n    ids.iter().all(|id| schema.name_by_field_id(*id).is_some())\n}","tryCatchPattern":"match list_scan_tasks(...) {\n    Err(e) if e.to_string().contains(\"not found in schema\") => {\n        // re-add the field or compact stale delete files\n    }\n    r => r,\n}","preventionTips":["Never drop columns used as equality-delete keys without compacting first","Track field-id stability across schema evolution","Validate delete-file schemas against current table schema"],"tags":["rust","iceberg","schema-evolution","field-id"],"backgroundTag":"resource-not-found","analyzedSha":"6469eb736d691e8e9b8a419a57edd6429ca77417","analyzedAt":"2026-09-11T21:06:21.487Z","contentChangedAt":"2026-09-11T21:06:21.487Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}