{"record":{"id":"d6f0dd229edc1a38","repo":"risingwavelabs/risingwave","slug":"input-data-file-has-multiple-live-position-dele","errorCode":null,"errorMessage":"input data file {} has multiple live position-delete files in snapshot {}","messagePattern":"input data file (.+?) has multiple live position-delete files in snapshot (.+?)","errorType":"validation","errorClass":"SinkError","httpStatus":null,"severity":"error","filePath":"src/stream/src/executor/iceberg_with_pk_index/compaction_resolver.rs","lineNumber":360,"sourceCode":"        // Only delete manifests can carry position-delete files.\n        if manifest_file.content != ManifestContentType::Deletes {\n            continue;\n        }\n        let manifest = manifest_file.load_manifest(file_io).await?;\n        for entry in manifest.entries() {\n            if !entry.is_alive() {\n                continue;\n            }\n            let data_file = entry.data_file();\n            if data_file.content_type() == DataContentType::PositionDeletes\n                && let Some(referenced) = data_file.referenced_data_file()\n                && input_paths.contains(referenced.as_str())\n            {\n                let positions = read_position_deletes_from_file(file_io, data_file)\n                    .await\n                    .map_err(SinkError::Iceberg)?;\n                if map.insert(referenced, positions).is_some() {\n                    return Err(SinkError::Iceberg(anyhow!(\n                        \"input data file {} has multiple live position-delete files in snapshot {}\",\n                        data_file.referenced_data_file().unwrap(),\n                        snapshot.snapshot_id()\n                    )));\n                }\n            }\n        }\n    }\n    Ok(map)\n}\n\n/// Builds a projection for PK root columns and maps the projected physical order back to the\n/// downstream PK order.\nfn pk_projection(\n    parquet_schema: &SchemaDescriptor,\n    pk_indices: &[usize],\n) -> Result<(ProjectionMask, Vec<usize>), SinkError> {\n    let root_count = parquet_schema.root_schema().get_fields().len();","sourceCodeStart":342,"sourceCodeEnd":378,"githubUrl":"https://github.com/risingwavelabs/risingwave/blob/6469eb736d691e8e9b8a419a57edd6429ca77417/src/stream/src/executor/iceberg_with_pk_index/compaction_resolver.rs#L342-L378","documentation":"While collecting input delete-vector files, the compaction resolver allows at most one live position-delete file per referenced data file. If `map.insert` finds a second position-delete file referencing the same data file in the snapshot, it fails with this `SinkError::Iceberg` because duplicate delete files are ambiguous and would double-apply deletes.","triggerScenarios":"An Iceberg snapshot whose manifest lists two or more live position-delete files whose `referenced_data_file` points at the same input data file included in `input_paths`, encountered in `collect_input_dvs` at src/stream/src/executor/iceberg_with_pk_index/compaction_resolver.rs:360.","commonSituations":"Corrupted or externally mutated Iceberg tables; compaction bugs that left stale duplicate delete files; third-party writers producing multiple delete files for one data file in a single snapshot.","solutions":["Run Iceberg table maintenance/expiring snapshots (e.g. `expire_snapshots`, `remove_orphan_files`) to clean duplicate delete files.","Check for concurrent writers or a buggy external engine producing duplicate position-delete files and fix the writer.","If caused by an internal compaction bug, report it with the snapshot id from the error and restore the table from a prior valid snapshot."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"// Before processing, scan the snapshot manifests and count position-delete\n// files per referenced data file; abort early if any count > 1.\nlet mut refs: HashMap<String, usize> = HashMap::new();\nfor df in position_delete_files(snapshot) {\n    *refs.entry(df.referenced_data_file().unwrap().into()).or_default() += 1;\n}\nassert!(refs.values().all(|&c| c == 1), \"duplicate position-delete files\");","typeGuard":null,"tryCatchPattern":"match collect_input_dvs(&snapshot).await {\n    Err(SinkError::Iceberg(e))\n        if e.to_string().contains(\"multiple live position-delete files\") => {\n        // run expire_snapshots / remove_orphan_files, then retry resolution\n    }\n    other => other?,\n}","preventionTips":["Run regular Iceberg maintenance: expire_snapshots and remove_orphan_files.","Avoid multiple concurrent writers appending position-delete files without a compaction strategy.","Audit third-party writers to confirm they never emit two live delete files for one data file."],"tags":["iceberg","compaction-resolver","position-delete","data-integrity"],"backgroundTag":"schema-validation-failed","analyzedSha":"6469eb736d691e8e9b8a419a57edd6429ca77417","analyzedAt":"2026-09-11T21:06:21.487Z","contentChangedAt":"2026-09-11T21:06:21.487Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}