risingwavelabs/risingwave · error · anyhow::Error

should get start response but get {:?}

Error message

should get start response but get {:?}

What it means

Same call as above: after sending UpdateVnodeRequest, SinkCoordinateClient::update_vnode_bitmap expects the next response to be a StartResponse. Any other CoordinateResponse variant triggers this error, indicating protocol desynchronization or a coordinator-side failure surfaced as a different message.

Source

Thrown at src/rpc_client/src/sink_coordinate_client.rs:135

    pub async fn update_vnode_bitmap(&mut self, vnode_bitmap: &Bitmap) -> anyhow::Result<u64> {
        self.send_request(CoordinateRequest {
            msg: Some(coordinate_request::Msg::UpdateVnodeRequest(
                UpdateVnodeBitmapRequest {
                    vnode_bitmap: Some(vnode_bitmap.to_protobuf()),
                },
            )),
        })
        .await?;
        match self.next_response().await? {
            CoordinateResponse {
                msg:
                    Some(coordinate_response::Msg::StartResponse(StartCoordinationResponse {
                        log_store_rewind_start_epoch,
                    })),
            } => Ok(log_store_rewind_start_epoch
                .ok_or_else(|| anyhow!("should get start epoch after update vnode bitmap"))?),
            msg => Err(anyhow!("should get start response but get {:?}", msg)),
        }
    }

    pub async fn stop(mut self) -> anyhow::Result<()> {
        self.send_request(CoordinateRequest {
            msg: Some(coordinate_request::Msg::Stop(true)),
        })
        .await?;
        match self.next_response().await? {
            CoordinateResponse {
                msg: Some(coordinate_response::Msg::Stopped(_)),
            } => Ok(()),
            msg => Err(anyhow!("should get Stopped but get {:?}", msg)),
        }
    }

    pub async fn align_initial_epoch(&mut self, initial_epoch: u64) -> anyhow::Result<u64> {
        self.send_request(CoordinateRequest {

View on GitHub (pinned to 6469eb736d)

Solutions

  1. Read the debug-printed `msg` to see what the coordinator returned
  2. Re-establish the sink coordination stream and retry from the failed epoch
  3. Check coordinator logs for restarts/errors; align component versions
Defensive patterns

Strategy: retry

Try / catch

// Rust
if let Err(e) = client.update_vnode_bitmap(&bitmap).await {
    if e.to_string().contains("start response") {
        // desync: recreate stream and retry once
        let mut c = SinkCoordinateClient::start(...).await?;
        c.update_vnode_bitmap(&bitmap).await?;
    } else { return Err(e); }
}

Prevention

When it happens

Trigger: Calling update_vnode_bitmap when the coordinator replies with something other than StartResponse — e.g. a response belonging to a stale request, coordinator restart mid-round, or mismatched protocol versions.

Common situations: Interleaved/misordered requests on the coordination stream; stream engine restarted during sink execution; upgrade skew between components.

Related errors


AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/32773dc80dff7876. Report an issue: GitHub.