{"record":{"id":"1c21d0485a923bbb","repo":"risingwavelabs/risingwave","slug":"no-snapshot-found-with-id-snapshot-id-while-publ","errorCode":null,"errorMessage":"No snapshot found with ID {snapshot_id} while publishing COW compaction","messagePattern":"No snapshot found with ID (.+?) while publishing COW compaction","errorType":"exception","errorClass":"HummockError","httpStatus":null,"severity":"error","filePath":"src/storage/src/hummock/compactor/iceberg_compaction/iceberg_compactor_runner.rs","lineNumber":547,"sourceCode":"                &cow_publish_plan,\n                data_files,\n            )\n            .await?;\n        }\n\n        Ok((stats, None))\n    }\n}\n\nasync fn live_data_files_for_snapshot(\n    table: &Table,\n    snapshot_id: i64,\n) -> HummockResult<Vec<DataFile>> {\n    let snapshot = table\n        .metadata()\n        .snapshot_by_id(snapshot_id)\n        .ok_or_else(|| {\n            HummockError::compaction_executor(anyhow::anyhow!(\n                \"No snapshot found with ID {snapshot_id} while publishing COW compaction\"\n            ))\n        })?;\n\n    let manifest_list = table\n        .object_cache()\n        .get_manifest_list(snapshot, &table.metadata_ref())\n        .await\n        .map_err(|e| HummockError::compaction_executor(e.as_report()))?;\n    let mut data_files = vec![];\n\n    for manifest_file in manifest_list\n        .entries()\n        .iter()\n        .filter(|entry| entry.has_added_files() || entry.has_existing_files())\n    {\n        let manifest = manifest_file\n            .load_manifest(table.file_io())","sourceCodeStart":529,"sourceCodeEnd":565,"githubUrl":"https://github.com/risingwavelabs/risingwave/blob/6469eb736d691e8e9b8a419a57edd6429ca77417/src/storage/src/hummock/compactor/iceberg_compaction/iceberg_compactor_runner.rs#L529-L565","documentation":"When publishing a copy-on-write (COW) compaction, the runner re-reads the data files of the snapshot the plan was built against via table.metadata().snapshot_by_id(snapshot_id). If that snapshot is no longer present in the current table metadata (e.g. it was expired by retention cleanup in the meantime), publishing cannot proceed and the task fails with this error.","triggerScenarios":"live_data_files_for_snapshot is called (from live_data_files_for_branch or build_cow_publish_data_files) with a snapshot_id captured during planning, but the table metadata no longer contains that snapshot because snapshot expiration ran between planning and publish.","commonSituations":"Long-running COW compaction that outlived the snapshot expiry window; aggressive expire_snapshots jobs or another compaction/commit advanced and cleaned metadata concurrently; misconfigured snapshot retention (very low max-snapshot-age / min-snapshots-to-keep) on the Iceberg table.","solutions":["Increase snapshot retention on the Iceberg table (e.g. history.expire.max-snapshot-age-ms / min-snapshots-to-keep) so COW compaction can finish before expiry.","Re-run the compaction — it will plan against the current snapshot instead of the expired one.","Reduce snapshot-expiry frequency or pause concurrent expire_snapshots jobs while COW compaction is in progress.","If the snapshot disappears immediately, verify no external tool (Spark/Flink/Iceberg maintenance) is expiring snapshots on the same table concurrently with RisingWave."],"exampleFix":"// before: aggressive retention\n// history.expire.max-snapshot-age-ms = 300000\n\n// after: keep enough snapshots/history for COW compaction\n// history.expire.max-snapshot-age-ms = 86400000\n// history.expire.min-snapshots-to-keep = 10","handlingStrategy":"validation","validationCode":"fn snapshot_exists(table: &Table, snapshot_id: i64) -> bool {\n    table.metadata().snapshot_by_id(snapshot_id).is_some()\n}","typeGuard":null,"tryCatchPattern":"match live_data_files_for_snapshot(&table, snapshot_id).await {\n    Ok(files) => publish(files),\n    Err(e) if e.to_string().contains(\"No snapshot found\") => {\n        // snapshot expired mid-compaction: re-plan against current snapshot\n        replan_and_retry().await\n    }\n    Err(e) => return Err(e),\n}","preventionTips":["Set generous snapshot retention (max-snapshot-age-ms, min-snapshots-to-keep) on tables used by RisingWave COW compaction.","Don't schedule external expire_snapshots jobs concurrently with RisingWave COW compaction.","Alert on long-running COW compaction tasks that approach the snapshot expiry window."],"tags":["rust","iceberg","snapshot","copy-on-write","compaction"],"backgroundTag":"record-not-found","analyzedSha":"6469eb736d691e8e9b8a419a57edd6429ca77417","analyzedAt":"2026-09-11T21:06:21.487Z","contentChangedAt":"2026-09-11T21:06:21.487Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}