risingwavelabs/risingwave · error · HummockError

No snapshot found with ID {snapshot_id} while publishing COW

Error message

No snapshot found with ID {snapshot_id} while publishing COW compaction

What it means

When publishing a copy-on-write (COW) compaction, the runner re-reads the data files of the snapshot the plan was built against via table.metadata().snapshot_by_id(snapshot_id). If that snapshot is no longer present in the current table metadata (e.g. it was expired by retention cleanup in the meantime), publishing cannot proceed and the task fails with this error.

Source

Thrown at src/storage/src/hummock/compactor/iceberg_compaction/iceberg_compactor_runner.rs:547

                &cow_publish_plan,
                data_files,
            )
            .await?;
        }

        Ok((stats, None))
    }
}

async fn live_data_files_for_snapshot(
    table: &Table,
    snapshot_id: i64,
) -> HummockResult<Vec<DataFile>> {
    let snapshot = table
        .metadata()
        .snapshot_by_id(snapshot_id)
        .ok_or_else(|| {
            HummockError::compaction_executor(anyhow::anyhow!(
                "No snapshot found with ID {snapshot_id} while publishing COW compaction"
            ))
        })?;

    let manifest_list = table
        .object_cache()
        .get_manifest_list(snapshot, &table.metadata_ref())
        .await
        .map_err(|e| HummockError::compaction_executor(e.as_report()))?;
    let mut data_files = vec![];

    for manifest_file in manifest_list
        .entries()
        .iter()
        .filter(|entry| entry.has_added_files() || entry.has_existing_files())
    {
        let manifest = manifest_file
            .load_manifest(table.file_io())

View on GitHub (pinned to 6469eb736d)

Solutions

1. Increase snapshot retention on the Iceberg table (e.g. history.expire.max-snapshot-age-ms / min-snapshots-to-keep) so COW compaction can finish before expiry.
2. Re-run the compaction — it will plan against the current snapshot instead of the expired one.
3. Reduce snapshot-expiry frequency or pause concurrent expire_snapshots jobs while COW compaction is in progress.
4. If the snapshot disappears immediately, verify no external tool (Spark/Flink/Iceberg maintenance) is expiring snapshots on the same table concurrently with RisingWave.

Example fix

// before: aggressive retention
// history.expire.max-snapshot-age-ms = 300000

// after: keep enough snapshots/history for COW compaction
// history.expire.max-snapshot-age-ms = 86400000
// history.expire.min-snapshots-to-keep = 10

fn snapshot_exists(table: &Table, snapshot_id: i64) -> bool {
    table.metadata().snapshot_by_id(snapshot_id).is_some()
}

match live_data_files_for_snapshot(&table, snapshot_id).await {
    Ok(files) => publish(files),
    Err(e) if e.to_string().contains("No snapshot found") => {
        // snapshot expired mid-compaction: re-plan against current snapshot
        replan_and_retry().await
    }
    Err(e) => return Err(e),
}

When it happens

Trigger: live_data_files_for_snapshot is called (from live_data_files_for_branch or build_cow_publish_data_files) with a snapshot_id captured during planning, but the table metadata no longer contains that snapshot because snapshot expiration ran between planning and publish.

Common situations: Long-running COW compaction that outlived the snapshot expiry window; aggressive expire_snapshots jobs or another compaction/commit advanced and cleaned metadata concurrently; misconfigured snapshot retention (very low max-snapshot-age / min-snapshots-to-keep) on the Iceberg table.

Understand the failure class

Background: Record Not Found Errors: "not found", RecordNotFound, and "was not found" — what they mean and how to fix them — this error's family across 28 libraries.

Related errors

• bounded compaction is not supported for copy-on-write tasks
• COW compaction must produce at most one table-scoped plan, g
• Cannot find the snapshot id in the iceberg table.
• Manual iceberg compaction task {} for sink {} failed: {}
• No iceberg compactor available

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/1c21d0485a923bbb. Report an issue: GitHub.