risingwavelabs/risingwave · error · anyhow::Error

should get start epoch after update vnode bitmap

Error message

should get start epoch after update vnode bitmap

What it means

SinkCoordinateClient::update_vnode_bitmap sends an UpdateVnodeRequest and expects a StartResponse carrying a log_store_rewind_start_epoch. If that inner epoch field is None, the client errors: the coordinator acknowledged but did not supply the required start epoch needed to continue.

Source

Thrown at src/rpc_client/src/sink_coordinate_client.rs:134

    }

    pub async fn update_vnode_bitmap(&mut self, vnode_bitmap: &Bitmap) -> anyhow::Result<u64> {
        self.send_request(CoordinateRequest {
            msg: Some(coordinate_request::Msg::UpdateVnodeRequest(
                UpdateVnodeBitmapRequest {
                    vnode_bitmap: Some(vnode_bitmap.to_protobuf()),
                },
            )),
        })
        .await?;
        match self.next_response().await? {
            CoordinateResponse {
                msg:
                    Some(coordinate_response::Msg::StartResponse(StartCoordinationResponse {
                        log_store_rewind_start_epoch,
                    })),
            } => Ok(log_store_rewind_start_epoch
                .ok_or_else(|| anyhow!("should get start epoch after update vnode bitmap"))?),
            msg => Err(anyhow!("should get start response but get {:?}", msg)),
        }
    }

    pub async fn stop(mut self) -> anyhow::Result<()> {
        self.send_request(CoordinateRequest {
            msg: Some(coordinate_request::Msg::Stop(true)),
        })
        .await?;
        match self.next_response().await? {
            CoordinateResponse {
                msg: Some(coordinate_response::Msg::Stopped(_)),
            } => Ok(()),
            msg => Err(anyhow!("should get Stopped but get {:?}", msg)),
        }
    }

    pub async fn align_initial_epoch(&mut self, initial_epoch: u64) -> anyhow::Result<u64> {

View on GitHub (pinned to 6469eb736d)

Solutions

  1. Verify compute node and meta/coordinator versions match; upgrade if skewed
  2. Retry the update_vnode_bitmap call after re-establishing the coordination stream
  3. Inspect coordinator logs; file a bug if StartResponse consistently omits the epoch
Defensive patterns

Strategy: try-catch

Try / catch

// Rust
let epoch = match client.update_vnode_bitmap(&bitmap).await {
    Ok(e) => e,
    Err(e) if e.to_string().contains("start epoch") => {
        tracing::warn!("missing start epoch: {e:#}; retrying with fresh stream");
        let mut c = rebuild_sink_coordinate_client().await?;
        c.update_vnode_bitmap(&bitmap).await?
    }
    Err(e) => return Err(e),
};

Prevention

When it happens

Trigger: Calling update_vnode_bitmap and the coordinator returns StartResponse with `log_store_rewind_start_epoch` unset (None) — coordinator bug or version skew omitting the field.

Common situations: Newer compute node against older coordinator that doesn't set the field; coordinator logic bug after fragment rescheduling; corrupted response.

Understand the failure class

Background: "invalid response format", "malformed payload", "missing data field": when an API returns 200 but the response shape is wrong — this error's family across 23 libraries.

Related errors

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/802bbe54a7e0e4d2. Report an issue: GitHub.