What causes "connection to postgres closed" in neondatabase/neon?

Inspect postgres logs under /log for the termination reason (crash, OOM, admin command)

connection to postgres closed

Error message

connection to postgres closed

What it means

The compute_ctl availability checker connected to PostgreSQL via tokio-postgres and then found client.is_closed() == true, meaning the connection task already terminated before any query ran. The checker refuses to run the health-check INSERT because the socket is dead. It indicates the server closed the session between startup and first use rather than a connect-time refusal.

Source

Thrown at compute_tools/src/checker.rs:15

use anyhow::{Ok, Result, anyhow};
use tokio_postgres::NoTls;
use tracing::{error, instrument, warn};

use crate::compute::ComputeNode;

/// Update timestamp in a row in a special service table to check
/// that we can actually write some data in this particular timeline.
#[instrument(skip_all)]
pub async fn check_writability(compute: &ComputeNode) -> Result<()> {
    // Connect to the database.
    let conf = compute.get_tokio_conn_conf(Some("compute_ctl:availability_checker"));
    let (client, connection) = conf.connect(NoTls).await?;
    if client.is_closed() {
        return Err(anyhow!("connection to postgres closed"));
    }

    // The connection object performs the actual communication with the database,
    // so spawn it off to run on its own.
    tokio::spawn(async move {
        if let Err(e) = connection.await {
            error!("connection error: {}", e);
        }
    });

    let query = "
    INSERT INTO public.health_check VALUES (1, pg_catalog.now())
        ON CONFLICT (id) DO UPDATE
         SET updated_at = pg_catalog.now();";

    match client.simple_query(query).await {
        Result::Ok(result) => {
            if result.len() != 1 {

View on GitHub (pinned to 8f60b04da4)

Solutions

Inspect postgres logs under <pgdata>/log for the termination reason (crash, OOM, admin command)
Verify the postgres process is alive and accepting connections (pg_isready) and that max_connections is not exhausted
Retry the availability check after postgres stabilizes; compute_ctl re-runs the checker periodically
Check pod/VM memory limits and raise them if postgres is being killed

Example fix

// before
let (client, connection) = conf.connect(NoTls).await?;
if client.is_closed() {
    return Err(anyhow!("connection to postgres closed"));
}
// after: tolerate a just-closed session by retrying the whole connect
let (client, connection) = conf.connect(NoTls).await?;
if client.is_closed() {
    tracing::warn!("postgres connection closed immediately; retrying");
    return Err(anyhow!("connection to postgres closed")); // keep error, but surface in retry loop upstream
}

Defensive patterns

Strategy: retry

Validate before calling

// Cheap pre-checks before running the writability probe
if !pgdata_path.join("postmaster.pid").exists() { /* postgres not running */ }
let (client, conn) = conf.connect(NoTls).await?;
if client.is_closed() { /* retry connect instead of proceeding */ }

Try / catch

// In the checker loop: treat closed-connection as transient
match client.simple_query(query).await {
    Ok(_) => Ok(()),
    Err(e) if e.is_closed() => { tracing::warn!("pg connection lost, will retry"); Ok(()) } // checker re-runs
    Err(e) => Err(anyhow!("writability check failed: {e}")),
}

Prevention

Supervise postgres and let the checker retry on failure rather than treating one failure as fatal
Keep postgres logs (log_directory under pgdata) shipped so the close reason is visible
Set max_connections and memory limits with headroom so new sessions are not terminated at startup

When it happens

Trigger: check_writability() runs, conf.connect(NoTls).await succeeds, but the backend terminates immediately: postgres is crashing or shutting down, the session is killed on startup (out of connections, out of memory, terminator), or the server closes the socket during startup packet exchange.

Common situations: PostgreSQL OOM-killed or restarting due to a bad GUC while compute_ctl polls availability; max_connections exhausted so the new backend is terminated right after fork; a compute VM under memory pressure; postgres still in recovery/crash-loop during endpoint start.

Related errors

AI-assisted analysis of neondatabase/neon@8f60b04da4 (2026-08-16). Data as JSON: /api/errors/e6a7735011159553. Report an issue: GitHub.