neondatabase/neon · error

Failed to stop storage controller database

Error message

Failed to stop storage controller database

What it means

Stopping storcon runs pg_ctl stop on the shared storage_controller_db. If the stop fails and the follow-up pg_ctl status check says postgres is still running, the stop is reported failed — postgres could not be shut down.

Source

Thrown at control_plane/src/storage_controller.rs:744

                // There is another storage controller instance running, so we return
                // and leave the database running.
                return Ok(());
            }
        }

        let pg_data_path = self.env.base_data_dir.join("storage_controller_db");

        println!("Stopping storage controller database...");
        let pg_stop_args = ["-D", &pg_data_path.to_string_lossy(), "stop"];
        let stop_status = self.pg_ctl(pg_stop_args).await;
        if !stop_status.success() {
            match self.is_postgres_running().await {
                Ok(false) => {
                    println!("Storage controller database is already stopped");
                    return Ok(());
                }
                Ok(true) => {
                    anyhow::bail!("Failed to stop storage controller database");
                }
                Err(err) => {
                    anyhow::bail!("Failed to stop storage controller database: {err}");
                }
            }
        }

        Ok(())
    }

    async fn is_postgres_running(&self) -> anyhow::Result<bool> {
        let pg_data_path = self.env.base_data_dir.join("storage_controller_db");

        let pg_status_args = ["-D", &pg_data_path.to_string_lossy(), "status"];
        let status_exitcode = self.pg_ctl(pg_status_args).await;

        // pg_ctl status returns this exit code if postgres is not running: in this case it is
        // fine that stop failed.  Otherwise it is an error that stop failed.

View on GitHub (pinned to 8f60b04da4)

Solutions

1. Stop all storage controller instances/processes before stopping the shared database
2. Retry the stop; escalate to immediate/fast shutdown mode if clean stop keeps failing
3. Inspect storage_controller_db/log/ for what blocked shutdown

let stop = pg_ctl(["-D", datadir, "stop"]).await;
if !stop.success() && is_postgres_running().await? {
    // retry a clean stop once, then escalate
    let _ = pg_ctl(["-D", datadir, "stop", "-m", "fast"]).await;
}

When it happens

Trigger: postgres ignores clean shutdown because a backend (e.g. a still-connected storcon instance) blocks it, a slow checkpoint on shutdown, or shutdown timeouts exceeded.

Common situations: Stopping the database while another storcon instance is still running against it, busy test teardown where connections are not closed, slow disks prolonging shutdown.

Related errors

• Failed to stop storage controller database: {err}
• Failed to start postgres {}
• pg_ctl status returned unexpected status code: {:?}
• pg_ctl status returned no status code
• pg_ctl failed, exit code: {}, stdout: {}, stderr: {}

AI-assisted analysis of neondatabase/neon@8f60b04da4 (2026-08-16). Data as JSON: /api/errors/fed9aebf6b280469. Report an issue: GitHub.