risingwavelabs/risingwave · error

should get the first epoch

Error message

should get the first epoch

What it means

In the non-vnode-bitmap branch of KvLogStoreReader::init, the reader awaits a oneshot receiver for the first epoch from the writer. The `.await` failing means the writer's oneshot sender was dropped without sending — the writer terminated or was dropped before initializing. Without the first epoch the reader cannot establish its epoch baseline.

Source

Thrown at src/stream/src/common/log_store_impl/kv_log_store/reader.rs:328

                            .map_err(|_| anyhow!("historical read semaphore closed"))?,
                    )
                } else {
                    None
                };
                KvLogStoreReaderFutureState::ReadStateStoreStream(
                    self.read_persisted_log_store(range_start).await?,
                    permit,
                )
            };
        self.rx.rewind(start_offset);
        Ok(())
    }

    async fn init(&mut self) -> LogStoreResult<()> {
        if let Some(init_epoch_rx) = self.init_epoch_rx.take() {
            let init_epoch = init_epoch_rx
                .await
                .map_err(|_| anyhow!("should get the first epoch"))?;
            let first_write_epoch = init_epoch.curr;

            assert_eq!(
                self.first_write_epoch.replace(first_write_epoch),
                None,
                "should not init twice"
            );
        } else {
            let (new_vnode_bitmap, write_epoch) = self
                .update_vnode_bitmap_rx
                .recv()
                .await
                .ok_or_else(|| anyhow!("failed to receive update vnode"))?;
            self.state.serde.update_vnode_bitmap(new_vnode_bitmap);
            self.first_write_epoch = Some(write_epoch);
        };

        self.future_state = KvLogStoreReaderFutureState::Reset;

View on GitHub (pinned to 6469eb736d)

Solutions

  1. Check the writer-side logs for a panic/termination before init.
  2. Ensure writer `init()` is called on the paired writer before or concurrently with reader init.
  3. Keep the writer alive until the reader has completed its init handshake.
  4. In tests, call writer.init(epoch) before awaiting reader.init().

Example fix

// before: writer dropped before sending init epoch
drop(writer);
reader.init().await?;

// after: writer initializes first
writer.init(epoch).await?;
reader.init().await?;
Defensive patterns

Strategy: validation

Validate before calling

// ensure the writer side is alive before reader init
assert!(!writer_dropped(), "paired writer must init before reader");
writer.init(epoch).await?;
reader.init().await?;

Type guard

fn init_handshake_ready(init_epoch_rx: &Option<oneshot::Receiver<EpochPair>>) -> bool { init_epoch_rx.is_some() }

Try / catch

if let Err(e) = reader.init().await {
    if e.to_string().contains("should get the first epoch") {
        // writer died before handshake; rebuild pair
        rebuild_log_store_pair();
    }
    return Err(e);
}

Prevention

When it happens

Trigger: Reader `init()` with `init_epoch_rx = Some(...)` whose paired writer dropped `init_epoch_tx` (writer actor terminated, panicked, or was dropped before calling writer init).

Common situations: Writer actor failure before first barrier; recovery paths tearing down the writer while the reader still initializes; incorrectly constructed reader/writer pairs in custom log store setups or tests.

Related errors

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/7ad53b9113f620a9. Report an issue: GitHub.