databendlabs/databend · error

rename_database: src (db) should exist

Error message

rename_database: src (db) should exist

What it means

This panic comes from an unreachable!() guard in rename_database in the meta-service API impl. The code path only runs when the database rename transaction failed because the source database did not exist at the expected sequence, which the preceding db_has_to_exist() call should have already converted into a proper user-facing error. Hitting this unreachable!() means the meta-store returned an inconsistent state: the sequence check indicated the DB is missing, but the code expected it to exist. It is an internal invariant violation, not a normal error return.

Solutions

  1. Check whether another session concurrently dropped the source database and retry the rename after recreating/verifying it
  2. Inspect meta-store logs for the failed txn to confirm which database id/tenant was involved and whether a race occurred
  3. Report as a meta-service bug with the log context; the unreachable!() indicates broken assumptions, not user error
  4. Upgrade to a patched databend release where the rename path handles concurrent-drop races

Example fix

// before
} else {
    db_has_to_exist(old_seq_db_id.seq(), tenant_dbname, "rename_database: src (db)")?;
    unreachable!("rename_database: src (db) should exist")
}
// after
} else {
    db_has_to_exist(old_seq_db_id.seq(), tenant_dbname, "rename_database: src (db)")?;
    Err(ErrorCode::DatabaseNotExists(String::from(tenant_dbname)))
}
Defensive patterns

Strategy: validation

Validate before calling

// before renaming, verify the db still exists
let exists = client.database_exists(tenant, dbname).await?;
if !exists { return Err(ErrorCode::DatabaseNotExists(dbname)); }

Type guard

fn db_meta_present(seq_db_id: &SeqV<DatabaseMeta>) -> bool { seq_db_id.seq() != 0 }

Try / catch

match result { Err(ErrorCode::DatabaseNotExists(_)) => retry_or_report(), Ok(v) => v, // panic here indicates a meta-service bug: report upstream
 }

Prevention

When it happens

Trigger: Calling rename_database when the meta-store transaction returns a state where old_seq_db_id is empty (db apparently gone) even though the earlier db_has_to_exist check accepted it — e.g. concurrent drop of the database racing with the rename, or a corrupted/inconsistent meta data tree.

Common situations: A concurrent DROP DATABASE executes between the rename's existence check and the rename transaction commit; meta-node replicas with divergent state; manual edits or bugs in the meta data store.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

AI-assisted analysis of databendlabs/databend@288d84d76e (2026-09-11). Data as JSON: /api/errors/b8fecae08b5695f3. Report an issue: GitHub.

Appendix: source

Thrown at src/meta/api/src/api_impl/database_api.rs:375

        let tenant_newdbname = DatabaseNameIdent::new(tenant_dbname.tenant(), &req.new_db_name);

        let mut trials = txn_backoff(None, func_name!());
        loop {
            trials.next().unwrap()?.await;

            // get old db, not exists return err
            let old_seq_db_id = self.get_pb(tenant_dbname).await?;

            let Some(old_seq_db_id) = old_seq_db_id else {
                if req.if_exists {
                    return Ok(RenameDatabaseReply {});
                } else {
                    db_has_to_exist(
                        old_seq_db_id.seq(),
                        tenant_dbname,
                        "rename_database: src (db)",
                    )?;
                    unreachable!("rename_database: src (db) should exist")
                }
            };

            let old_db_id = old_seq_db_id.data;

            let old_seq_db_meta = self.get_pb(&old_db_id).await?;

            db_has_to_exist(
                old_seq_db_meta.seq(),
                tenant_dbname,
                "rename_database: src (db)",
            )?;

            debug!(
                old_db_id :? = old_db_id,
                tenant_dbname :? =(tenant_dbname);
                "rename_database"
            );

View on GitHub (pinned to 288d84d76e)