risingwavelabs/risingwave · error

streaming job {job_id} not found in streaming_job_databases

Error message

streaming job {job_id} not found in streaming_job_databases

What it means

When assembling all_fragments grouped by database, render_actors_with_allocator looks up the database for the current job in streaming_job_databases. If the job has no database mapping in the loaded context, this error is returned with the job ID.

Source

Thrown at src/meta/src/controller/scale.rs:1076

                            vnode_bitmap,
                            splits: splits.remove(&actor_id).unwrap_or_default(),
                        },
                    )
                })
                .collect();

            let fragment = InflightFragmentInfo {
                fragment_id,
                distribution_type,
                fragment_type_mask,
                vnode_count,
                nodes: stream_node.clone(),
                actors,
                state_table_ids: state_table_ids.clone(),
            };

            let &database_id = streaming_job_databases.get(&job_id).ok_or_else(|| {
                anyhow!("streaming job {job_id} not found in streaming_job_databases")
            })?;

            all_fragments
                .entry(database_id)
                .or_default()
                .entry(job_id)
                .or_default()
                .insert(fragment_id, fragment);
        }
    }

    Ok(all_fragments)
}

pub(crate) struct EnsembleActorTemplate {
    assignment: BTreeMap<WorkerId, BTreeMap<u32, Option<Bitmap>>>,
    distribution_type: DistributionType,
    actor_count: u32,

View on GitHub (pinned to 6469eb736d)

Solutions

1. Ensure streaming_job_databases is populated for every job before calling the renderer (use into_database_contexts, which derives it from the catalog).
2. Backfill the job's database mapping in the meta catalog for legacy jobs.
3. Rebuild the loaded context via the standard load_fragment_context / load_fragment_context_for_jobs entry points instead of hand-constructing it.
4. Retry after the upgrade/migration completes fully.

Example fix

// before
let context = build_loaded_context(...);
render_actors_with_allocator(&context, ...);
// after
let contexts = context.into_database_contexts(); // fills streaming_job_databases per database
for ctx in contexts.values() {
    render_actors_with_allocator(ctx, ...)?;
}

for job_id in ctx.job_map.keys() {
    assert!(ctx.streaming_job_databases.contains_key(job_id),
            "job {job_id} has no database mapping");
}

fn has_database_mapping(ctx: &LoadedFragmentContext, job_id: JobId) -> bool {
    ctx.streaming_job_databases.contains_key(&job_id)
}

let Some(&database_id) = ctx.streaming_job_databases.get(&job_id) else {
    return Err(anyhow!("job {job_id} missing database mapping; rebuild context"));
};

When it happens

Trigger: render_actors_with_allocator is invoked with a job_id that exists in job_map/fragments but was absent from streaming_job_databases — e.g. the context was built by a code path that skipped the job-to-database mapping, or the job predates database scoping and its mapping was never backfilled.

Common situations: Upgrades from pre-multi-database versions with un-mapped legacy jobs; a caller constructing LoadedFragmentContext manually without filling streaming_job_databases; catalog rows missing the database association.

Related errors

• streaming job {job_id} not found
• fragments {:?} not found
• streaming jobs {:?} not found
• entry fragments {:?} have inconsistent parallelism settings
• Multiple jobs found in no-shuffle ensemble

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/89cea70986e7eb00. Report an issue: GitHub.