risingwavelabs/risingwave · error

failed to find handle {} to start

Error message

failed to find handle {} to start

What it means

`CoordinationHandleManager::start` is asked to start a set of writer handles by `HandleId`, but the id is not present in `self.writer_handles`. Since every valid handle is registered in `start_handle` (via `next_event`) before it can be started, a missing id means the caller passed stale or fabricated handle ids — an internal bookkeeping inconsistency between the caller (e.g. `wait_init_handles`) and the manager's registry.

Source

Thrown at src/meta/src/manager/sink_coordination/coordinator_worker.rs:293

struct CoordinationHandleManager {
    param: SinkParam,
    writer_handles: HashMap<HandleId, SinkWriterCoordinationHandle>,
    next_handle_id: HandleId,
    request_rx: UnboundedReceiver<SinkWriterCoordinationHandle>,
}

impl CoordinationHandleManager {
    fn start(
        &mut self,
        log_store_rewind_start_epoch: Option<u64>,
        handle_ids: impl IntoIterator<Item = HandleId>,
    ) -> anyhow::Result<()> {
        for handle_id in handle_ids {
            let handle = self
                .writer_handles
                .get_mut(&handle_id)
                .ok_or_else(|| anyhow!("failed to find handle {} to start", handle_id,))?;
            handle.start(log_store_rewind_start_epoch).map_err(|_| {
                anyhow!(
                    "failed to start {:?} for handle {}",
                    log_store_rewind_start_epoch,
                    handle_id
                )
            })?;
        }
        Ok(())
    }

    fn ack_aligned_initial_epoch(&mut self, aligned_initial_epoch: u64) -> anyhow::Result<()> {
        for (handle_id, handle) in &mut self.writer_handles {
            handle
                .ack_aligned_initial_epoch(aligned_initial_epoch)
                .map_err(|_| {
                    anyhow!(
                        "failed to ack aligned initial epoch {:?} for handle {}",

View on GitHub (pinned to 6469eb736d)

Solutions

1. Log the full set of keys in `writer_handles` versus the requested `handle_ids` to identify the stale id.
2. Ensure `wait_init_handles` collects handle ids only from `next_event` results (live registrations).
3. If the sink job is being recovered, re-register the writer handles before calling `start`.
4. Check whether a concurrent `alter_parallelisms`/stop path removed the handle while start was in flight and add synchronization.

Example fix

// before
let handle_ids: Vec<HandleId> = persisted_ids.clone();
manager.start(handle_ids, epoch)?;

// after
let handle_ids: Vec<HandleId> = manager
    .registered_handle_ids() // ids taken from live writer_handles
    .collect();
manager.start(handle_ids, epoch)?;

// Validate handle ids before calling start
let registered: HashSet<_> = manager.registered_handle_ids().collect();
let missing: Vec<_> = handle_ids.iter().filter(|id| !registered.contains(id)).collect();
anyhow::ensure!(missing.is_empty(), "unregistered handle ids: {:?}", missing);

fn is_registered(mgr: &CoordinationHandleManager, id: &HandleId) -> bool {
    mgr.registered_handle_ids().any(|rid| rid == *id)
}

match manager.start(handle_ids, epoch) {
    Ok(()) => {},
    Err(e) if e.to_string().contains("failed to find handle") => {
        warn!("stale handle ids; re-collecting from live registrations");
    }
    Err(e) => return Err(e),
}

When it happens

Trigger: Calling `start(handle_ids, log_store_rewind_start_epoch)` with a `HandleId` that was never inserted into `writer_handles`, or one that was already consumed/removed (e.g. after `Stop`), or a resurrected job whose handles were dropped on a previous error path.

Common situations: Sink job recovery after meta-node restart where persisted handle ids no longer match live registrations; race between parallelism change (dropping handles) and a concurrent start; bugs in handle-id persistence/migration code.

Understand the failure class

Background: Record Not Found Errors: "not found", RecordNotFound, and "was not found" — what they mean and how to fix them — this error's family across 28 libraries.

Related errors

• failed to find handle {} when acknowledging the commit for e
• infinite
• failed to start {:?} for handle {}
• failed to ack aligned initial epoch {:?} for handle {}
• failed to acknowledge the commit for epoch {} on handle {}

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/f7a8bf659f623d0c. Report an issue: GitHub.