risingwavelabs/risingwave · error · TimeTravel

invalid time travel delta chain: delta {} has prev version {

Error message

invalid time travel delta chain: delta {} has prev version {}, but replay has reached {}

What it means

During time-travel archive replay, each HummockVersionDelta must extend the chain monotonically: the replay cursor (last_version) must never have moved past the delta's prev_id. If d.prev_id < last_version.id, the archive is corrupted, deltas were applied out of order, or a non-time-travel delta leaked into the archive, so replay aborts instead of producing a bogus historical version.

Source

Thrown at src/meta/src/hummock/manager/time_travel.rs:856

}

/// The `HummockVersion` is actually `InHummockVersion`. It requires `refill_version`.
fn replay_archive(
    version: PbHummockVersion,
    deltas: impl Iterator<Item = PbHummockVersionDelta>,
) -> Result<HummockVersion> {
    // The pb version ann pb version delta are actually written by InHummockVersion and InHummockVersionDelta, respectively.
    // Using HummockVersion make it easier for `refill_version` later.
    let mut last_version = HummockVersion::from_persisted_protobuf_owned(version);
    for d in deltas {
        let d = HummockVersionDelta::from_persisted_protobuf_owned(d);
        debug_assert!(
            !should_mark_next_time_travel_version_snapshot(&d),
            "unexpected time travel delta {:?}",
            d
        );
        if d.prev_id < last_version.id {
            return Err(Error::TimeTravel(anyhow!(format!(
                "invalid time travel delta chain: delta {} has prev version {}, but replay has reached {}",
                d.id, d.prev_id, last_version.id
            ))));
        }
        // Compaction deltas are not included in the time travel archive, so there may be gaps
        // between the last replayed version and this delta's previous version.
        last_version.id = d.prev_id;
        last_version.apply_version_delta(&d);
    }
    Ok(last_version)
}

pub fn require_sql_meta_store_err() -> Error {
    Error::TimeTravel(anyhow!("require SQL meta store"))
}

/// Time travel delta replay only expect `NewL0SubLevel`. In all other cases, a new version snapshot should be created.
pub fn should_mark_next_time_travel_version_snapshot(delta: &HummockVersionDelta) -> bool {

View on GitHub (pinned to 6469eb736d)

Solutions

1. Inspect the time-travel archive deltas around the failing delta id and verify they are ordered by epoch/prev_id.
2. Check that only NewL0SubLevel deltas are archived (should_mark_next_time_travel_version_snapshot must be false for each archived delta).
3. Rebuild the time-travel archive from scratch if it is corrupt; replay cannot skip or reorder deltas.
4. If reproducible, file a bug with the delta id, prev_id and reached version id from the message.

Example fix

// before: replay assumes arbitrary archive order
let last_version = replay_archive(deltas);
// after: pre-sort deltas by prev_id/epoch and drop duplicates before replay
let mut deltas: Vec<_> = deltas;
deltas.sort_by_key(|d| (d.prev_id, d.id));
deltas.dedup_by_key(|d| d.id);
let last_version = replay_archive(deltas);

// Rust: before replay, ensure chain monotonicity
fn deltas_are_chained(deltas: &[HummockVersionDelta]) -> bool {
    let mut reached = base_version_id;
    deltas.iter().all(|d| d.prev_id >= reached && { reached = d.id; true })
}

match replay_archive(&deltas) {
    Ok(v) => /* use v */,
    Err(e) if e.to_string().contains("invalid time travel delta chain") => /* quarantine archive, rebuild from scratch */,
    Err(e) => return Err(e),
}

When it happens

Trigger: replay_archive(), called from epoch_to_version and the test test_replay_archive_delta_chain, when iterating archived deltas and encountering a delta whose prev_id is older than the version already replayed.

Common situations: Corrupted or hand-edited time-travel archive tables; a delta archived twice or out of epoch order; compaction/time-travel delta classification bugs (the debug_assert above also guards against unexpected time travel deltas slipping in).

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• time-travel version expired: table {table_id}, epoch {epoch}
• time travel
• require SQL meta store
• Cannot find the snapshot id in the iceberg table.
• inconsistent hummock version: expected {}, actual {}

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/871af2e4875afe19. Report an issue: GitHub.