{"record":{"id":"fed9aebf6b280469","repo":"neondatabase/neon","slug":"failed-to-stop-storage-controller-database","errorCode":null,"errorMessage":"Failed to stop storage controller database","messagePattern":"Failed to stop storage controller database","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"control_plane/src/storage_controller.rs","lineNumber":744,"sourceCode":"                // There is another storage controller instance running, so we return\n                // and leave the database running.\n                return Ok(());\n            }\n        }\n\n        let pg_data_path = self.env.base_data_dir.join(\"storage_controller_db\");\n\n        println!(\"Stopping storage controller database...\");\n        let pg_stop_args = [\"-D\", &pg_data_path.to_string_lossy(), \"stop\"];\n        let stop_status = self.pg_ctl(pg_stop_args).await;\n        if !stop_status.success() {\n            match self.is_postgres_running().await {\n                Ok(false) => {\n                    println!(\"Storage controller database is already stopped\");\n                    return Ok(());\n                }\n                Ok(true) => {\n                    anyhow::bail!(\"Failed to stop storage controller database\");\n                }\n                Err(err) => {\n                    anyhow::bail!(\"Failed to stop storage controller database: {err}\");\n                }\n            }\n        }\n\n        Ok(())\n    }\n\n    async fn is_postgres_running(&self) -> anyhow::Result<bool> {\n        let pg_data_path = self.env.base_data_dir.join(\"storage_controller_db\");\n\n        let pg_status_args = [\"-D\", &pg_data_path.to_string_lossy(), \"status\"];\n        let status_exitcode = self.pg_ctl(pg_status_args).await;\n\n        // pg_ctl status returns this exit code if postgres is not running: in this case it is\n        // fine that stop failed.  Otherwise it is an error that stop failed.","sourceCodeStart":726,"sourceCodeEnd":762,"githubUrl":"https://github.com/neondatabase/neon/blob/8f60b04da47ffefe0e52bda2440134b42874eb75/control_plane/src/storage_controller.rs#L726-L762","documentation":"Stopping storcon runs pg_ctl stop on the shared storage_controller_db. If the stop fails and the follow-up pg_ctl status check says postgres is still running, the stop is reported failed — postgres could not be shut down.","triggerScenarios":"postgres ignores clean shutdown because a backend (e.g. a still-connected storcon instance) blocks it, a slow checkpoint on shutdown, or shutdown timeouts exceeded.","commonSituations":"Stopping the database while another storcon instance is still running against it, busy test teardown where connections are not closed, slow disks prolonging shutdown.","solutions":["Stop all storage controller instances/processes before stopping the shared database","Retry the stop; escalate to immediate/fast shutdown mode if clean stop keeps failing","Inspect storage_controller_db/log/ for what blocked shutdown"],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"let stop = pg_ctl([\"-D\", datadir, \"stop\"]).await;\nif !stop.success() && is_postgres_running().await? {\n    // retry a clean stop once, then escalate\n    let _ = pg_ctl([\"-D\", datadir, \"stop\", \"-m\", \"fast\"]).await;\n}","preventionTips":["Stop all storcon instances before stopping the shared database","Retry the stop before declaring failure","Use immediate mode only after fast stop fails"],"tags":["rust","storage-controller","postgres","pg-ctl","shutdown"],"backgroundTag":"postgres-stop-failed","analyzedSha":"8f60b04da47ffefe0e52bda2440134b42874eb75","analyzedAt":"2026-08-16T23:39:28.135Z","schemaVersion":2},"datasetVersion":"2026-08-17T04:17:16.089Z"}