{"record":{"id":"ec7d9e1c3145605b","repo":"risingwavelabs/risingwave","slug":"data-file-should-not-in-task-deletes","errorCode":null,"errorMessage":"Data file should not in task deletes","messagePattern":"Data file should not in task deletes","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"src/connector/src/source/iceberg/mod.rs","lineNumber":390,"sourceCode":"        let mut equality_delete_files = vec![];\n        let mut equality_delete_files_set = HashSet::new();\n        let mut equality_delete_ids = None;\n        let mut scan_builder = table.scan().snapshot_id(snapshot_id).select_all();\n        if predicate != IcebergPredicate::AlwaysTrue {\n            scan_builder = scan_builder.with_filter(predicate.clone());\n        }\n        let scan = scan_builder.build()?;\n        let file_scan_stream = scan.plan_files().await?;\n\n        #[for_await]\n        for task in file_scan_stream {\n            let task: FileScanTask = task?;\n\n            // Collect delete files for separate scan types, but keep task.deletes intact\n            for delete_file in &task.deletes {\n                match delete_file.file_type {\n                    iceberg::spec::DataContentType::Data => {\n                        bail!(\"Data file should not in task deletes\");\n                    }\n                    iceberg::spec::DataContentType::EqualityDeletes => {\n                        if equality_delete_files_set.insert(delete_file.file_path.clone()) {\n                            if equality_delete_ids.is_none() {\n                                equality_delete_ids = delete_file.equality_ids.clone();\n                            } else if equality_delete_ids != delete_file.equality_ids {\n                                bail!(\n                                    \"The schema of iceberg equality delete file must be consistent\"\n                                );\n                            }\n                            equality_delete_files.push(delete_file.to_file_scan_task(&task));\n                        }\n                    }\n                    iceberg::spec::DataContentType::PositionDeletes => {\n                        if position_delete_files_set.insert(delete_file.file_path.clone()) {\n                            position_delete_files.push(delete_file.to_file_scan_task(&task));\n                        }\n                    }","sourceCodeStart":372,"sourceCodeEnd":408,"githubUrl":"https://github.com/risingwavelabs/risingwave/blob/6469eb736d691e8e9b8a419a57edd6429ca77417/src/connector/src/source/iceberg/mod.rs#L372-L408","documentation":"While listing Iceberg scan tasks, the code iterates each task's delete files and dispatches on their content type. A `DataContentType::Data` file must never appear in `task.deletes` (deletes should only contain EqualityDeletes or PositionDeletes). If it does, the table metadata/scan planning produced an unexpected state, so the code bails as an internal invariant violation.","triggerScenarios":"`list_scan_tasks`/`list_scan_tasks_inner` receiving a `FileScanTask` whose `deletes` vector contains a file with `file_type == DataContentType::Data` — indicative of a bug or corrupt/unexpected metadata from the iceberg-rust library version in use.","commonSituations":"Upgrading/changing the iceberg-rust dependency so delete files are classified differently; corrupt table metadata; a merge-on-read plan that incorrectly includes data files in the delete set.","solutions":["Inspect the table metadata for the offending data file wrongly listed as a delete file.","Check the pinned iceberg-rust crate version for known bugs in FileScanTask delete classification; upgrade/downgrade accordingly.","Verify the snapshot/manifest being scanned is not corrupt by re-running with a different snapshot.","If reproducible, file/inspect how deletes are populated in list_scan_tasks_inner and add filtering or fix the dependency."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"let bad = tasks.iter().flat_map(|t| t.deletes.iter()).any(|d| d.file_type == iceberg::spec::DataContentType::Data);\nif bad { /* skip or regenerate scan tasks */ }","typeGuard":null,"tryCatchPattern":"match list_scan_tasks(...) {\n    Ok(r) => r,\n    Err(e) if e.to_string().contains(\"Data file should not in task deletes\") => {\n        // re-plan with a different snapshot or report corrupt metadata\n        ...\n    }\n    Err(e) => return Err(e),\n}","preventionTips":["Keep the iceberg-rust dependency version aligned with RisingWave's expectations","Validate table metadata after upgrades","Avoid scanning snapshots produced by unusual write paths"],"tags":["rust","iceberg","scan-planning","invariant"],"backgroundTag":"internal-invariant-violation","analyzedSha":"6469eb736d691e8e9b8a419a57edd6429ca77417","analyzedAt":"2026-09-11T21:06:21.487Z","contentChangedAt":"2026-09-11T21:06:21.487Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}