risingwavelabs/risingwave · error

failed to start {:?} for handle {}

Error message

failed to start {:?} for handle {}

What it means

The handle exists in `writer_handles`, but its `start(log_store_rewind_start_epoch)` call returned `Err`. The manager discards the handle's own error and wraps it in this generic message, so the failure happened inside the coordination handle's start routine — typically rewinding/joining the writer to the given log-store epoch or spawning its task.

Source

Thrown at src/meta/src/manager/sink_coordination/coordinator_worker.rs:295

    param: SinkParam,
    writer_handles: HashMap<HandleId, SinkWriterCoordinationHandle>,
    next_handle_id: HandleId,
    request_rx: UnboundedReceiver<SinkWriterCoordinationHandle>,
}

impl CoordinationHandleManager {
    fn start(
        &mut self,
        log_store_rewind_start_epoch: Option<u64>,
        handle_ids: impl IntoIterator<Item = HandleId>,
    ) -> anyhow::Result<()> {
        for handle_id in handle_ids {
            let handle = self
                .writer_handles
                .get_mut(&handle_id)
                .ok_or_else(|| anyhow!("failed to find handle {} to start", handle_id,))?;
            handle.start(log_store_rewind_start_epoch).map_err(|_| {
                anyhow!(
                    "failed to start {:?} for handle {}",
                    log_store_rewind_start_epoch,
                    handle_id
                )
            })?;
        }
        Ok(())
    }

    fn ack_aligned_initial_epoch(&mut self, aligned_initial_epoch: u64) -> anyhow::Result<()> {
        for (handle_id, handle) in &mut self.writer_handles {
            handle
                .ack_aligned_initial_epoch(aligned_initial_epoch)
                .map_err(|_| {
                    anyhow!(
                        "failed to ack aligned initial epoch {:?} for handle {}",
                        aligned_initial_epoch,
                        handle_id

View on GitHub (pinned to 6469eb736d)

Solutions

1. Preserve and log the inner `Err` from `handle.start` (change `map_err(|_| ...)` to include the source) to see the root cause.
2. Verify `log_store_rewind_start_epoch` is still available in the log store and not truncated.
3. Ensure the writer handle's task/channel is alive before starting; recreate the handle if it was dropped.
4. Avoid double-starting the same handle across recovery attempts; check for idempotency in `handle.start`.

Example fix

// before
handle.start(log_store_rewind_start_epoch).map_err(|_| {
    anyhow!("failed to start {:?} for handle {}", log_store_rewind_start_epoch, handle_id)
})?;

// after
handle.start(log_store_rewind_start_epoch)
    .with_context(|| format!("failed to start {:?} for handle {}", log_store_rewind_start_epoch, handle_id))?;

// Check the epoch is still retained in the log store before starting
anyhow::ensure!(
    log_store.contains_epoch(log_store_rewind_start_epoch),
    "rewind epoch {} no longer retained",
    log_store_rewind_start_epoch
);

if let Err(e) = manager.start(ids, rewind_epoch) {
    error!(error = ?e, "handle start failed; will retry after re-registration");
    // re-register handles and retry once before failing the job
}

When it happens

Trigger: Calling `start` on a registered handle whose underlying writer channel/task is already closed, or where the log store cannot rewind to `log_store_rewind_start_epoch` (epoch no longer retained), or the handle was already started/stopped.

Common situations: Recovery after a barrier/log-store truncation removed the requested start epoch; writer actor already exited due to an earlier error; double start of the same sink job during failover.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• log_store_rewind_start_epoch {} not later than first_epoch {
• failed to ack aligned initial epoch {:?} for handle {}
• initial epoch {} greater than aligned initial epoch {}
• since_timestamp requires at least one upstream table
• The cluster is recovering

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/1003b022d541136c. Report an issue: GitHub.