What causes "failed to acknowledge the commit for epoch {} on handle {}" in risingwavelabs/risingwave?

Include the inner error in the wrapper (use `with_context` instead of `map_err(|_| ...)`) to expose the real cause.

failed to acknowledge the commit for epoch {} on handle {}

Error message

failed to acknowledge the commit for epoch {} on handle {}

What it means

The handle exists, but its `ack_commit(epoch)` returned `Err`; the manager wraps this into the given message, discarding the inner cause. Typical root causes are the writer actor being gone (channel closed) or the epoch not matching the writer's expectations (e.g. acking an epoch that was not prepared or was already acked).

Source

Thrown at src/meta/src/manager/sink_coordination/coordinator_worker.rs:334

        }
        Ok(())
    }

    fn ack_commit(
        &mut self,
        epoch: u64,
        handle_ids: impl IntoIterator<Item = HandleId>,
    ) -> anyhow::Result<()> {
        for handle_id in handle_ids {
            let handle = self.writer_handles.get_mut(&handle_id).ok_or_else(|| {
                anyhow!(
                    "failed to find handle {} when acknowledging the commit for epoch {}",
                    handle_id,
                    epoch
                )
            })?;
            handle.ack_commit(epoch).map_err(|_| {
                anyhow!(
                    "failed to acknowledge the commit for epoch {} on handle {}",
                    epoch,
                    handle_id
                )
            })?;
        }
        Ok(())
    }

    async fn next_request_inner(
        writer_handles: &mut HashMap<HandleId, SinkWriterCoordinationHandle>,
    ) -> anyhow::Result<(HandleId, coordinate_request::Msg)> {
        poll_fn(|cx| {
            for (handle_id, handle) in writer_handles.iter_mut() {
                if let Poll::Ready(result) = handle.poll_next_request(cx) {
                    return Poll::Ready(result.map(|request| (*handle_id, request)));
                }
            }

View on GitHub (pinned to 6469eb736d)

Solutions

Include the inner error in the wrapper (use `with_context` instead of `map_err(|_| ...)`) to expose the real cause.
Verify the epoch being acked was prepared on that handle and not previously acked.
Check writer task health/logs for a crash preceding the failed ack.
Make `ack_commit` on the handle idempotent or tolerant of repeated epochs to survive event replays.

Example fix

// before
handle.ack_commit(epoch).map_err(|_| anyhow!("failed to acknowledge the commit for epoch {} on handle {}", epoch, handle_id))?;

// after
handle.ack_commit(epoch)
    .with_context(|| format!("failed to acknowledge the commit for epoch {} on handle {}", epoch, handle_id))?;

Defensive patterns

Strategy: try-catch

Validate before calling

// Only ack epochs that were prepared on this handle and not yet acked
anyhow::ensure!(
    prepared_epochs.contains(&epoch) && !acked_epochs.contains(&epoch),
    "epoch {} not prepared or already acked",
    epoch
);

Try / catch

match handle.ack_commit(epoch) {
    Ok(()) => {},
    Err(e) => {
        warn!(epoch, error = ?e, "commit ack failed; writer may have exited");
        // recreate handle or fail the sink job with the preserved cause
    }
}

Prevention

Make ack_commit idempotent for repeated epochs
Monitor writer actor health before commit acks
Preserve inner errors instead of map_err(|_| ...)

When it happens

Trigger: Calling `ack_commit` on a live-registered handle whose writer task exited, or acking an epoch out of order (not prepared / already committed), during commit processing in the coordination worker.

Common situations: Writer crashed mid-transaction so the commit ack cannot be delivered; duplicated commit events after a retry/failover; epochs committed after the handle was logically stopped but before removal.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/162c428bddfd3b1e. Report an issue: GitHub.