risingwavelabs/risingwave · error · anyhow::Error

empty commit metadata

Error message

empty commit metadata

What it means

A `CommitRequest` must carry commit `metadata` (e.g. serialized vnode bitmap and epoch state for the sink), and this error fires when `request.metadata` is None. Without it the coordinator cannot record the commit's metadata for recovery. It signals either a sender bug or an incompatible message version.

Source

Thrown at src/meta/src/manager/sink_coordination/handle.rs:137

                .ok_or_else(|| anyhow!("end of request stream"))?
                .map_err(anyhow::Error::from)?;
            let request = request.msg.ok_or_else(|| anyhow!("None msg in request"))?;
            match &request {
                coordinate_request::Msg::StartRequest(_)
                | coordinate_request::Msg::Stop(_)
                | coordinate_request::Msg::AlignInitialEpochRequest(_) => {}
                coordinate_request::Msg::CommitRequest(request) => {
                    if let Some(prev_epoch) = self.prev_epoch
                        && request.epoch < prev_epoch
                    {
                        return Poll::Ready(Err(anyhow!(
                            "invalid commit epoch {}, prev_epoch {}",
                            request.epoch,
                            prev_epoch
                        )));
                    }
                    if request.metadata.is_none() {
                        return Poll::Ready(Err(anyhow!("empty commit metadata")));
                    };
                    self.prev_epoch = Some(request.epoch);
                }
                coordinate_request::Msg::UpdateVnodeRequest(request) => {
                    let bitmap = Bitmap::from(
                        request
                            .vnode_bitmap
                            .as_ref()
                            .ok_or_else(|| anyhow!("empty vnode bitmap"))?,
                    );
                    self.vnode_bitmap = bitmap;
                }
            };
            request
        };
        Poll::Ready(result)
    }
}

View on GitHub (pinned to 6469eb736d)

Solutions

1. Upgrade all sink executors to a version that populates `CommitRequest.metadata`.
2. Fix the executor code path that builds the commit metadata so it always sets the field before sending.
3. Add executor-side validation to fail fast if metadata would be None rather than sending the request.
4. Confirm protobuf definitions are identical across components (no field renumbering/shadowing).

Example fix

// before
CoordinateRequest { msg: Some(coordinate_request::Msg::CommitRequest(CommitRequest { epoch, metadata: None })) }
// after
CoordinateRequest { msg: Some(coordinate_request::Msg::CommitRequest(CommitRequest { epoch, metadata: Some(metadata) })) }

// Executor side: refuse to send a commit without metadata
assert!(metadata.is_some(), "CommitRequest.metadata must be populated");

if let Err(e) = coordinator_next_request() {
    if e.to_string().contains("empty commit metadata") {
        tracing::error!("executor sent CommitRequest without metadata; check executor version");
    }
}

When it happens

Trigger: A `CommitRequest` with `metadata: None` is polled from the request stream after passing the epoch monotonicity check.

Common situations: Older sink executor version that predates the metadata field talking to a newer meta node; executor bug where metadata fails to serialize and is silently set to None; manually crafted or replayed messages in tests.

Understand the failure class

Background: "missing required argument" and "the following required arguments were not provided": what required-argument errors mean and how to fix them — this error's family across 20 libraries.

Related errors

• query_epoch not set in distributed lookup join
• empty sink metadata
• None msg in request
• invalid commit epoch {}, prev_epoch {}
• empty vnode bitmap

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/9f474444e05afa0a. Report an issue: GitHub.