risingwavelabs/risingwave · error

BUG: Worker not found for new actor {}

Error message

BUG: Worker not found for new actor {}

What it means

In `diff_fragment`, after computing `added_actor_ids` (actors present in the new plan but not the old), the code looks each new actor up in `curr_actors` to find its assigned worker. A new actor missing from that map means the diff and the current actor snapshot are inconsistent — an internal bug — hence the explicit "BUG:" prefix. This would otherwise leave added actors with no worker assignment in the produced reschedule commands.

Source

Thrown at src/meta/src/stream/scale.rs:674

    all_actor_dispatchers: HashMap<ActorId, Vec<PbDispatcher>>,
    job_extra_info: Option<&StreamingJobExtraInfo>,
) -> MetaResult<Reschedule> {
    let prev_ids: HashSet<_> = prev_fragment_info.actors.keys().cloned().collect();
    let curr_ids: HashSet<_> = curr_actors.keys().cloned().collect();

    let removed_actors: HashSet<_> = &prev_ids - &curr_ids;
    let added_actor_ids: HashSet<_> = &curr_ids - &prev_ids;
    let kept_ids: HashSet<_> = prev_ids.intersection(&curr_ids).cloned().collect();
    debug_assert!(
        kept_ids.is_empty(),
        "kept actors found in scale; expected full rebuild, prev={prev_ids:?}, curr={curr_ids:?}, kept={kept_ids:?}"
    );

    let mut added_actors = HashMap::new();
    for &actor_id in &added_actor_ids {
        let InflightActorInfo { worker_id, .. } = curr_actors
            .get(&actor_id)
            .ok_or_else(|| anyhow!("BUG: Worker not found for new actor {}", actor_id))?;

        added_actors
            .entry(*worker_id)
            .or_insert_with(Vec::new)
            .push(actor_id);
    }

    let mut vnode_bitmap_updates = HashMap::new();
    for actor_id in kept_ids {
        let prev_actor = &prev_fragment_info.actors[&actor_id];
        let curr_actor = &curr_actors[&actor_id];

        // Check if the vnode distribution has changed.
        if prev_actor.vnode_bitmap != curr_actor.vnode_bitmap
            && let Some(bitmap) = curr_actor.vnode_bitmap.clone()
        {
            vnode_bitmap_updates.insert(actor_id, bitmap);
        }

View on GitHub (pinned to 6469eb736d)

Solutions

1. Treat as a meta bug: capture the actor id from the message and report it with the reschedule that triggered it.
2. Retry the reschedule; a transient race may resolve once snapshots are consistent.
3. Check that no concurrent reschedule runs against the same job, which could desynchronize the snapshots.
4. Review the diff logic (added_actor_ids derivation vs curr_actors source) for a version/branch mismatch in how snapshots are collected.

// "BUG:" errors are not user-recoverable; capture diagnostics and retry once
match build_reschedule_from_context(ctx).await {
    Err(e) if e.to_string().contains("BUG: Worker not found") => {
        error!("meta invariant violated: {e}");
        sleep(backoff).await;
        build_reschedule_from_context(rebuild(ctx)).await
    }
    r => r,
}

When it happens

Trigger: Calling `build_reschedule_commands` where an added actor id exists in the diff result but not in the `curr_actors` map built from the current fragment states — e.g. the diff was computed against a different snapshot than the actor lookup, or `added_actor_ids` derivation is buggy.

Common situations: Concurrent modification of the fragment state between diff computation and actor lookup; a meta internal bug in actor-id bookkeeping during scale-out planning; testing with mocked render results that don't align with curr_actors.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• previous fragment info for {} not found
• previous fragment info for {fragment_id} not found
• fragment {} not found in previous state
• request record count {} not match the response record count
• should not fail because we have set `data` and `partition_ke

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/e8c06e1bbfde3998. Report an issue: GitHub.