risingwavelabs/risingwave · error

fragments {:?} not found

Error message

fragments {:?} not found

What it means

load_fragment_context fetches all Fragment rows referenced by the given no-shuffle ensembles. If the DB returns fewer fragments than the ensemble components require, the missing fragment IDs are reported with this anyhow error. It guarantees the renderer never proceeds with partially-known topology.

Source

Thrown at src/meta/src/controller/scale.rs:484

        .flat_map(|ensemble| ensemble.components.iter().copied())
        .collect();

    let fragment_models = Fragment::find()
        .filter(fragment::Column::FragmentId.is_in(required_fragment_ids.iter().copied()))
        .all(txn)
        .await?;

    let found_fragment_ids: HashSet<_> = fragment_models
        .iter()
        .map(|fragment| fragment.fragment_id)
        .collect();

    if found_fragment_ids.len() != required_fragment_ids.len() {
        let missing = required_fragment_ids
            .difference(&found_fragment_ids)
            .copied()
            .collect_vec();
        return Err(anyhow!("fragments {:?} not found", missing).into());
    }

    let fragment_models: HashMap<_, _> = fragment_models
        .into_iter()
        .map(|fragment| (fragment.fragment_id, fragment))
        .collect();

    let job_ids: HashSet<_> = fragment_models
        .values()
        .map(|fragment| fragment.job_id)
        .collect();

    if job_ids.is_empty() {
        return Ok(LoadedFragmentContext::default());
    }

    let jobs: HashMap<_, _> = StreamingJob::find()
        .filter(streaming_job::Column::JobId.is_in(job_ids.iter().copied().collect_vec()))

View on GitHub (pinned to 6469eb736d)

Solutions

1. Re-fetch fresh ensemble data (re-run the load stage against the current catalog) instead of using cached/stale ensembles.
2. Identify the listed missing fragment IDs and confirm whether their job was dropped; if so, exclude those ensembles or cancel the rescale request.
3. Retry the rescale when no conflicting DDL is in flight.
4. If the fragments should exist, check meta store consistency/backup.

let existing: HashSet<_> = Fragment::find()
    .filter(fragment::Column::FragmentId.is_in(required_ids.iter().copied()))
    .all(txn).await?
    .into_iter().map(|f| f.fragment_id).collect();
let missing: Vec<_> = required_ids.difference(&existing).collect();
if !missing.is_empty() { return Err(anyhow!("stale ensembles, missing fragments {missing:?}")); }

match load_fragment_context(txn, ensembles).await {
    Ok(ctx) => proceed(ctx),
    Err(e) if e.to_string().contains("not found") => refresh_ensembles_and_retry(),
    Err(e) => return Err(e),
}

When it happens

Trigger: Calling load_fragment_context with ensembles whose component fragment IDs no longer exist in the fragments table — e.g. rescale API invoked with stale ensemble info after the fragments were deleted by a job cancel/replace, or cross-job ensembles referencing foreign fragments.

Common situations: A scale/rescale request racing with DDL that drops fragments; meta store rollback leaving ensembles pointing at removed fragments; operator tooling replaying old ensemble snapshots.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors

• streaming jobs {:?} not found
• cannot find StreamActor of actor {}
• Some streaming jobs already exist in meta, please start with
• fragment id {} from actor {} is different from fragment {}
• Failed to retrieve fragment description: fragment {} (job_id

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/ac992096f2416f16. Report an issue: GitHub.