risingwavelabs/risingwave · error · MetaError

no active streaming workers for reschedule

Error message

no active streaming workers for reschedule

What it means

build_reschedule_from_context refuses to build a ReschedulePlan when the provided worker_nodes map is empty, returning `no active streaming workers for reschedule`. A reschedule needs live compute nodes to place fragments on; with zero candidates there is no valid plan, so the intent fails fast.

Source

Thrown at src/meta/src/barrier/worker.rs:263

                }
            }
        }
        _ => {
            new_barrier.command = Some((command, notifier));
            Ok(Some(new_barrier))
        }
    }
}

fn build_reschedule_from_context(
    env: &MetaSrvEnv,
    worker_nodes: HashMap<WorkerId, WorkerNode>,
    database_id: DatabaseId,
    context: RescheduleContext,
    database_info: &InflightDatabaseInfo,
) -> MetaResult<Option<crate::barrier::ReschedulePlan>> {
    if worker_nodes.is_empty() {
        return Err(anyhow!("no active streaming workers for reschedule").into());
    }

    if context.is_empty() {
        return Ok(None);
    }

    // Barrier worker resolves this intent against a stable in-flight snapshot.
    // Reuse the same fragment view for preview comparison and command building.
    let all_prev_fragments = database_info
        .fragment_infos()
        .map(|fragment| (fragment.fragment_id, fragment))
        .collect();

    let previewed = preview_actor_assignments(&worker_nodes, &context.loaded)?;

    if rendered_layout_matches_current(&previewed.fragments, &all_prev_fragments)? {
        return Ok(None);
    }

View on GitHub (pinned to 6469eb736d)

Solutions

1. Ensure compute nodes are running and registered: check `SHOW CLUSTERS;` / worker liveness and restart failed compute nodes.
2. Verify the deployment profile includes compute nodes (e.g. risedev profile with compute-node entries).
3. After adding nodes, trigger recovery so pending reschedule intents are retried against the new worker set.
4. Check meta logs for worker deregistration (heartbeat loss) that emptied the worker map.

Example fix

// before: rescheduling without checking worker availability
plan = build_reschedule_from_context(env, workers, db, ctx, info)?;
// after: bail out with a clear precondition
ensure!(!workers.is_empty(), "start compute nodes before requesting reschedule");

// refuse to reschedule with no live workers
let workers = env.worker_nodes().await?;
if workers.is_empty() {
    return Err(anyhow!("start compute nodes before rescheduling"));
}

fn has_active_workers(workers: &HashMap<WorkerId, WorkerNode>) -> bool {
    !workers.is_empty()
}

match err {
    e if e.to_string().contains("no active streaming workers") => {
        start_compute_nodes().await?; // then trigger recovery to retry the intent
    }
    e => return Err(e.into()),
}

When it happens

Trigger: resolve_reschedule_intent forwards an empty worker_nodes HashMap — i.e. the cluster has no active streaming worker nodes at the moment a reschedule (scale/fragment placement) is requested or a barrier triggers recomputation.

Common situations: All compute nodes crashed or were drained/decommissioned while streaming jobs still need placement; startup ordering where meta is up before any compute node registers; misconfigured risedev profile omitting compute nodes.

Understand the failure class

Background: EmptyResultError / "no results found": when an API or scraper succeeds but returns zero rows — this error's family across 9 libraries.

Related errors

• reschedule failed
• no active frontend nodes found
• no frontend worker available
• invalid parallelism
• no workers to assign; assignment is meaningless

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/c4c3f8046fc34c96. Report an issue: GitHub.