risingwavelabs/risingwave · error

current epoch does not match with decoded epoch

Error message

current epoch {} does not match with decoded epoch {}

What it means

Thrown during log store replay/deserialization: while scanning buffered rows, the in-memory current tracking state (`curr_epoch` in `BarrierAligning`/`AllConsumingRow`) does not match the `decoded_epoch` of the row just decoded from the log store. The replay logic must see rows grouped by epoch; a decoded row from a different epoch in the middle of processing means the persisted data or the state machine is inconsistent.

Solutions

  1. Check whether the state table/log store backend (e.g. the metadata KV store) was corrupted or manually modified; restore from a clean snapshot/backup.
  2. Verify the replaying cluster uses the same version that wrote the log data (encoding/serialization compatibility).
  3. If it occurs after crash recovery, retry recovery from the latest Hummock checkpoint; if reproducible, report with the persisted key range involved.
  4. Inspect decoded row bytes at the failing point for truncated or misaligned payloads.

Example fix

// before: mixing old-version log data into new cluster
rw meta start --state-store hummock+... (pointing at foreign data)
// after: restore only data produced by the same cluster and version
rw meta start --state-store hummock+s3://my-bucket (original cluster data)
Defensive patterns

Strategy: try-catch

Validate before calling

// Before recovery, sanity-check the metadata KV store range for the stream
// (e.g. iterate rows and confirm epochs are non-decreasing and contiguous)
for (key, val) in log_store.scan_range(start..end) {
    let row = KvLogStoreRow::parse(key, val)?;
    // compare with previously decoded epoch; abort early on mismatch
}

Type guard

fn decoded_epoch_matches(state_epoch: u64, row: &KvLogStoreRow) -> bool {
    state_epoch == row.epoch()
}

Try / catch

// Rust
match log_store.replay(...).await {
    Err(e) if e.to_string().contains("does not match with decoded epoch") => {
        // treat as corrupted log: restore snapshot and retry recovery from last checkpoint
    }
    r => r?,
}

Prevention

When it happens

Trigger: During replay of the KV log store (`replay`/`next_barrier` style iteration), when a decoded row's epoch differs from `curr_epoch` tracked in `StreamState` for the currently aligned vnode set.

Common situations: Corrupted or partially written log store rows (e.g. crash between writes), manual tampering/copying of the metadata KV store, replaying a log written by a different RisingWave version with different row encoding, or mixing data from an old cluster into a new one.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors


AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/c2df8a06593bd97a. Report an issue: GitHub.

Appendix: source

Thrown at src/stream/src/common/log_store_impl/kv_log_store/serde.rs:1080

                            epoch: decoded_epoch,
                            size: read_size,
                            op: AlignedLogStoreOp::Barrier {
                                vnodes: Arc::new(aligned_vnodes.finish()),
                                is_checkpoint,
                            },
                        }));
                    } else {
                        match &mut self.stream_state {
                            StreamState::BarrierAligning {
                                aligned_vnodes,
                                read_size,
                                curr_epoch,
                                is_checkpoint: current_is_checkpoint,
                            } => {
                                aligned_vnodes.set(vnode.to_index(), true);
                                *read_size += size;
                                if curr_epoch != &decoded_epoch {
                                    return Err(anyhow!(
                                        "current epoch {} does not match with decoded epoch {}",
                                        curr_epoch,
                                        decoded_epoch
                                    ));
                                }
                                if current_is_checkpoint != &is_checkpoint {
                                    return Err(anyhow!(
                                        "current is_checkpoint {} does not match with decoded is_checkpoint {}",
                                        current_is_checkpoint,
                                        is_checkpoint
                                    ));
                                }
                            }
                            other => {
                                let mut aligned_vnodes =
                                    BitmapBuilder::zeroed(self.serde.vnodes().len());
                                aligned_vnodes.set(vnode.to_index(), true);
                                *other = StreamState::BarrierAligning {

View on GitHub (pinned to 6469eb736d)