risingwavelabs/risingwave · error

receive commit request on epoch {} from handle {} during alt

Error message

receive commit request on epoch {} from handle {} during alter parallelism

What it means

During `alter_parallelisms` the coordinator is renegotiating parallelism and expects only Stop/NewHandle/Stop events from handles. Receiving a `CommitRequest` means a sink writer tried to commit data on some epoch mid-renegotiation, which would corrupt the parallelism-change protocol, so the coordinator bails out with this error.

Source

Thrown at src/meta/src/manager/sink_coordination/coordinator_worker.rs:484

            .filter(|handle_id| !requests.handle_ids.contains(handle_id))
            .cloned()
            .collect();
        while !remaining_handles.is_empty() || !requests.aligned() {
            let (handle_id, event) = self.next_event().await?;
            match event {
                CoordinationHandleManagerEvent::NewHandle => {
                    requests.add_new_request(handle_id, (), self.vnode_bitmap(handle_id))?;
                }
                CoordinationHandleManagerEvent::UpdateVnodeBitmap => {
                    assert!(remaining_handles.remove(&handle_id));
                    requests.add_new_request(handle_id, (), self.vnode_bitmap(handle_id))?;
                }
                CoordinationHandleManagerEvent::Stop => {
                    assert!(remaining_handles.remove(&handle_id));
                    self.stop_handle(handle_id)?;
                }
                CoordinationHandleManagerEvent::CommitRequest { epoch, .. } => {
                    bail!(
                        "receive commit request on epoch {} from handle {} during alter parallelism",
                        epoch,
                        handle_id
                    );
                }
                CoordinationHandleManagerEvent::AlignInitialEpoch(epoch) => {
                    bail!(
                        "receive AlignInitialEpoch on epoch {} from handle {} during alter parallelism",
                        epoch,
                        handle_id
                    );
                }
            }
        }
        Ok(requests.handle_ids)
    }
}

View on GitHub (pinned to 6469eb736d)

Solutions

1. Confirm the writer honors the Stop request before it can emit further commits; check the writer's event loop for buffered commits sent after Stop was queued.
2. Retry the ALTER (e.g. alter sink parallelism) after the sink has quiesced; the coordinator aborts safely on this error.
3. Investigate ordering between control channel and commit channel on the writer side — commits must be gated behind the parallelism-alignment ack.
4. Check logs for which handle_id committed and whether it is a leftover handle that should have been stopped first.

// before altering parallelism, verify the sink is quiesced
// (no pending commits) via the coordinator metrics/logs

fn is_during_alter(phase: &Phase) -> bool { matches!(phase, Phase::AlteringParallelisms) }

// retry the alter statement after the error; coordinator state is safely aborted
if err.contains("during alter parallelism") { sleep(backoff); retry_alter(); }

When it happens

Trigger: A sink writer that has not yet been stopped (and not yet restarted with the new parallelism) sends a commit request while `alter_parallelisms` is in progress — e.g. the writer did not receive/process the Stop event, or a new handle became active before the coordinator finished the alignment phase.

Common situations: Sink writers lagging behind coordinator control messages (slow network, backpressure); a writer restarted with stale state that resumes committing immediately; parallelism changes performed while heavy commit traffic is in flight.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• should have been handled
• receive AlignInitialEpoch on epoch {} from handle {} during
• receive AlignInitialEpoch after initialization
• parallelism should not be zero
• infinite

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/0c065be8060e8f84. Report an issue: GitHub.