neondatabase/neon · error

Failed to read storcon pid file at {pid_file:?}: {err}

Error message

Failed to read storcon pid file at {pid_file:?}: {err}

What it means

When stopping one storage controller instance, storcon inspects every OTHER instance's storage_controller.pid file to decide whether the shared database must stay up. The read has no NotFound handling — any read error, including a missing pid file, is fatal to the stop command.

Source

Thrown at control_plane/src/storage_controller.rs:719

    pub async fn stop(&self, stop_args: NeonStorageControllerStopArgs) -> anyhow::Result<()> {
        background_process::stop_process(
            stop_args.immediate,
            COMMAND,
            &self.pid_file(stop_args.instance_id),
        )?;

        let storcon_instances = self.env.storage_controller_instances().await?;
        for (instance_id, instanced_dir_path) in storcon_instances {
            if instance_id == stop_args.instance_id {
                continue;
            }

            let pid_file = instanced_dir_path.join("storage_controller.pid");
            let pid = tokio::fs::read_to_string(&pid_file)
                .await
                .map_err(|err| {
                    anyhow::anyhow!("Failed to read storcon pid file at {pid_file:?}: {err}")
                })?
                .parse::<i32>()
                .expect("pid is valid i32");

            let other_proc_alive = !background_process::process_has_stopped(Pid::from_raw(pid))?;
            if other_proc_alive {
                // There is another storage controller instance running, so we return
                // and leave the database running.
                return Ok(());
            }
        }

        let pg_data_path = self.env.base_data_dir.join("storage_controller_db");

        println!("Stopping storage controller database...");
        let pg_stop_args = ["-D", &pg_data_path.to_string_lossy(), "stop"];
        let stop_status = self.pg_ctl(pg_stop_args).await;
        if !stop_status.success() {

View on GitHub (pinned to 8f60b04da4)

Solutions

1. Restore or remove the stale instance directory so its pid file can be read or the instance is fully gone
2. Stop or clean up the other storcon instances (via the CLI) before stopping this one

Example fix

// before
let pid = tokio::fs::read_to_string(&pid_file).await
    .map_err(|err| anyhow::anyhow!("Failed to read storcon pid file at {pid_file:?}: {err}"))?;
// after — treat a missing pid file as "not running"
let pid = match tokio::fs::read_to_string(&pid_file).await {
    Ok(s) => s.parse::<i32>()?,
    Err(e) if e.kind() == std::io::ErrorKind::NotFound => continue,
    Err(e) => anyhow::bail!("Failed to read storcon pid file at {pid_file:?}: {e}"),
};

for (_, dir) in other_instances {
    let pid_file = dir.join("storage_controller.pid");
    match tokio::fs::read_to_string(&pid_file).await {
        Ok(s) => { let _pid: i32 = s.parse().context("malformed pid file")?; }
        Err(e) if e.kind() == std::io::ErrorKind::NotFound => continue, // no pid => not running
        Err(e) => anyhow::bail!("unreadable pid file {pid_file:?}: {e}"),
    }
}

When it happens

Trigger: Another storcon instance directory exists under the env but its storage_controller.pid is missing (deleted manually, instance never fully started) or unreadable (permissions), so stop cannot determine whether the database is still needed.

Common situations: Manually pruning instance dirs or pid files, mixed-version envs whose instances never wrote pid files, crashed instances that left a directory without a pid file.

Related errors

• Failed to read startup script: {e}
• Failed to stop storage controller database
• Failed to stop storage controller database: {err}
• `datadir` must be a directory when calling this function: {d
• {} with pid {} did not stop in {:?} seconds

AI-assisted analysis of neondatabase/neon@8f60b04da4 (2026-08-16). Data as JSON: /api/errors/2d4c5e78cff45756. Report an issue: GitHub.