{"record":{"id":"b8fecae08b5695f3","repo":"databendlabs/databend","slug":"rename-database-src-db-should-exist","errorCode":null,"errorMessage":"rename_database: src (db) should exist","messagePattern":"rename_database: src \\(db\\) should exist","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"src/meta/api/src/api_impl/database_api.rs","lineNumber":375,"sourceCode":"        let tenant_newdbname = DatabaseNameIdent::new(tenant_dbname.tenant(), &req.new_db_name);\n\n        let mut trials = txn_backoff(None, func_name!());\n        loop {\n            trials.next().unwrap()?.await;\n\n            // get old db, not exists return err\n            let old_seq_db_id = self.get_pb(tenant_dbname).await?;\n\n            let Some(old_seq_db_id) = old_seq_db_id else {\n                if req.if_exists {\n                    return Ok(RenameDatabaseReply {});\n                } else {\n                    db_has_to_exist(\n                        old_seq_db_id.seq(),\n                        tenant_dbname,\n                        \"rename_database: src (db)\",\n                    )?;\n                    unreachable!(\"rename_database: src (db) should exist\")\n                }\n            };\n\n            let old_db_id = old_seq_db_id.data;\n\n            let old_seq_db_meta = self.get_pb(&old_db_id).await?;\n\n            db_has_to_exist(\n                old_seq_db_meta.seq(),\n                tenant_dbname,\n                \"rename_database: src (db)\",\n            )?;\n\n            debug!(\n                old_db_id :? = old_db_id,\n                tenant_dbname :? =(tenant_dbname);\n                \"rename_database\"\n            );","sourceCodeStart":357,"sourceCodeEnd":393,"githubUrl":"https://github.com/databendlabs/databend/blob/288d84d76e20a2f8f7173bda9691eb6ece301aa9/src/meta/api/src/api_impl/database_api.rs#L357-L393","documentation":"This panic comes from an unreachable!() guard in rename_database in the meta-service API impl. The code path only runs when the database rename transaction failed because the source database did not exist at the expected sequence, which the preceding db_has_to_exist() call should have already converted into a proper user-facing error. Hitting this unreachable!() means the meta-store returned an inconsistent state: the sequence check indicated the DB is missing, but the code expected it to exist. It is an internal invariant violation, not a normal error return.","triggerScenarios":"Calling rename_database when the meta-store transaction returns a state where old_seq_db_id is empty (db apparently gone) even though the earlier db_has_to_exist check accepted it — e.g. concurrent drop of the database racing with the rename, or a corrupted/inconsistent meta data tree.","commonSituations":"A concurrent DROP DATABASE executes between the rename's existence check and the rename transaction commit; meta-node replicas with divergent state; manual edits or bugs in the meta data store.","solutions":["Check whether another session concurrently dropped the source database and retry the rename after recreating/verifying it","Inspect meta-store logs for the failed txn to confirm which database id/tenant was involved and whether a race occurred","Report as a meta-service bug with the log context; the unreachable!() indicates broken assumptions, not user error","Upgrade to a patched databend release where the rename path handles concurrent-drop races"],"exampleFix":"// before\n} else {\n    db_has_to_exist(old_seq_db_id.seq(), tenant_dbname, \"rename_database: src (db)\")?;\n    unreachable!(\"rename_database: src (db) should exist\")\n}\n// after\n} else {\n    db_has_to_exist(old_seq_db_id.seq(), tenant_dbname, \"rename_database: src (db)\")?;\n    Err(ErrorCode::DatabaseNotExists(String::from(tenant_dbname)))\n}","handlingStrategy":"validation","validationCode":"// before renaming, verify the db still exists\nlet exists = client.database_exists(tenant, dbname).await?;\nif !exists { return Err(ErrorCode::DatabaseNotExists(dbname)); }","typeGuard":"fn db_meta_present(seq_db_id: &SeqV<DatabaseMeta>) -> bool { seq_db_id.seq() != 0 }","tryCatchPattern":"match result { Err(ErrorCode::DatabaseNotExists(_)) => retry_or_report(), Ok(v) => v, // panic here indicates a meta-service bug: report upstream\n }","preventionTips":["Avoid concurrent DROP and RENAME on the same database","Keep all meta nodes on the same databend version","Check meta-service logs for txn conflicts around the rename"],"tags":["rust","meta-service","internal-invariant","concurrency"],"backgroundTag":"internal-invariant-violation","analyzedSha":"288d84d76e20a2f8f7173bda9691eb6ece301aa9","analyzedAt":"2026-09-11T11:29:36.208Z","contentChangedAt":"2026-09-11T11:29:36.208Z","schemaVersion":2},"datasetVersion":"2026-09-16T04:17:20.429Z"}