risingwavelabs/risingwave · critical

database should exist for streaming job

Error message

database should exist for streaming job

What it means

In scale.rs, when building per-database resharding contexts, each streaming job's database_id must resolve to a DatabaseModel in database_map; .expect panics if not. It means a streaming job is recorded against a database that no longer exists in the catalog — catalog inconsistency.

Solutions

1. Check the database referenced by the job still exists: SELECT from the databases/objects tables for the database_id.
2. Drop or clean the orphaned streaming job(s) referencing the missing database, or restore the database.
3. Avoid concurrent DROP DATABASE and resharding operations; retry the scale operation after the catalog is consistent.

let missing: Vec<_> = streaming_job_databases.iter()
    .map(|(_, db)| db)
    .filter(|db| !database_map.contains_key(db))
    .collect();
if !missing.is_empty() { bail!("databases missing for jobs: {:?}", missing); }

When it happens

Trigger: Running resharding/ALTER ... SET PARALLELISM scale operations while the streaming_job_databases map references a database_id missing from database_map (database dropped while jobs remain, or stale catalog rows).

Common situations: Dropping a database with lingering jobs due to prior partial failure, meta store corruption, manual row deletion, or racing a DROP DATABASE with a scale operation.

Understand the failure class

Background: "Not found" and "does not exist" errors: why "Task not found", "No such folder", and "Can't find" fire when a lookup comes back empty — this error's family across 14 libraries.

Related errors

• Catalog error
• database not found when resolving reschedule intent
• failed to decode persisted `DefaultColumnDesc`
• should exist
• {0}

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/713d1a6aee031b7a. Report an issue: GitHub.

Appendix: source

Thrown at src/meta/src/controller/scale.rs:355

            ensembles,
            mut job_fragments,
            mut job_map,
            streaming_job_databases,
            mut database_map,
            mut fragment_source_ids,
            mut fragment_splits,
        } = self;

        let mut contexts = HashMap::<DatabaseId, Self>::new();
        let mut fragment_databases = HashMap::<FragmentId, DatabaseId>::new();
        let mut unresolved_ensembles = 0usize;
        let mut unresolved_ensemble_sample: Option<Vec<FragmentId>> = None;

        for (job_id, database_id) in streaming_job_databases {
            let context = contexts.entry(database_id).or_insert_with(|| {
                let database_model = database_map
                    .remove(&database_id)
                    .expect("database should exist for streaming job");
                Self {
                    ensembles: Vec::new(),
                    job_fragments: HashMap::new(),
                    job_map: HashMap::new(),
                    streaming_job_databases: HashMap::new(),
                    database_map: HashMap::from([(database_id, database_model)]),
                    fragment_source_ids: HashMap::new(),
                    fragment_splits: HashMap::new(),
                }
            });

            let fragments = job_fragments
                .remove(&job_id)
                .expect("job fragments should exist for streaming job");
            for fragment_id in fragments.keys().copied() {
                fragment_databases.insert(fragment_id, database_id);
                if let Some(source_id) = fragment_source_ids.remove(&fragment_id) {
                    context.fragment_source_ids.insert(fragment_id, source_id);

View on GitHub (pinned to 6469eb736d)