neondatabase/neon · critical

safekeeper connstrings should be provided

Error message

safekeeper connstrings should be provided

What it means

For a Primary-mode compute, compute_ctl requires a non-empty list of safekeepers: neither spec.safekeeper_connstrings nor the 'neon.safekeepers' GUC in cluster settings was present. Primary nodes synchronously replicate WAL to safekeepers and cannot start without them. Non-primary modes (e.g. read replicas) are allowed to have an empty list.

Source

Thrown at compute_tools/src/compute.rs:346

                    Some(ShardStripeSize(u32::from_str(&guc)?))
                } else {
                    None
                };
                pageserver_conninfo =
                    Some(PageserverConnectionInfo::from_connstr(&guc, stripe_size)?);
            }
        }
        let pageserver_conninfo = pageserver_conninfo.ok_or(anyhow::anyhow!(
            "pageserver connection information should be provided"
        ))?;

        // Similarly for safekeeper connection strings
        let safekeeper_connstrings = if spec.safekeeper_connstrings.is_empty() {
            if matches!(spec.mode, ComputeMode::Primary) {
                spec.cluster
                    .settings
                    .find("neon.safekeepers")
                    .ok_or(anyhow::anyhow!("safekeeper connstrings should be provided"))?
                    .split(',')
                    .map(|str| str.to_string())
                    .collect()
            } else {
                vec![]
            }
        } else {
            spec.safekeeper_connstrings.clone()
        };

        let storage_auth_token = spec.storage_auth_token.clone();
        let tenant_id: TenantId = if let Some(tenant_id) = spec.tenant_id {
            tenant_id
        } else {
            let guc = spec
                .cluster
                .settings
                .find("neon.tenant_id")

View on GitHub (pinned to 8f60b04da4)

Solutions

1. Add "neon.safekeepers": "host1:5678,host2:5678,..." to spec.cluster.settings for Primary mode
2. Or populate spec.safekeeper_connstrings at the control plane so the fallback is unnecessary
3. Verify spec.mode is actually Primary; replicas must not require safekeepers
4. Pre-validate the spec: Primary mode implies a non-empty safekeepers list

Example fix

// before: Primary spec without safekeepers -> error
// after
{
  "mode": "Primary",
  "cluster": { "settings": [
    { "name": "neon.safekeepers", "value": "safekeeper-0:5678,safekeeper-1:5678,safekeeper-2:5678" }
  ]}
}

// Primary mode requires safekeepers; validate before starting compute
if matches!(spec.mode, ComputeMode::Primary) {
    let has_sks = !spec.safekeeper_connstrings.is_empty()
        || spec.cluster.settings.find("neon.safekeepers").is_some();
    if !has_sks { anyhow::bail!("Primary spec without safekeepers"); }
}

fn primary_has_safekeepers(spec: &ComputeSpec) -> bool {
    !matches!(spec.mode, ComputeMode::Primary)
        || !spec.safekeeper_connstrings.is_empty()
        || spec.cluster.settings.find("neon.safekeepers").map(|s| !s.trim().is_empty()).unwrap_or(false)
}

// Config error: report and stop; retrying cannot succeed until the spec changes
if let Err(e) = ParsedSpec::try_from(spec) {
    if e.chain().any(|c| c.to_string().contains("safekeeper")) {
        eprintln!("spec is missing neon.safekeepers; add a comma-separated host:port list");
    }
    return Err(e);
}

When it happens

Trigger: ParsedSpec::try_from with spec.mode == ComputeMode::Primary, spec.safekeeper_connstrings empty, and settings.find("neon.safekeepers") returning None. Note the GUC value must be a comma-separated host:port list; an empty-string GUC also fails downstream when split/joined.

Common situations: Hand-written or generated spec for a primary endpoint missing the safekeepers GUC; cplane version that stopped populating safekeeper_connstrings; a spec intended for a replica accidentally marked Primary; typo in the GUC name so find() misses it.

Related errors

• pageserver connection information should be provided
• tenant id should be provided
• timeline id should be provided
• shard {shard_index} missing from pageserver_connection_info
• postgres --sync-safekeepers exited with non-zero status: {}.

AI-assisted analysis of neondatabase/neon@8f60b04da4 (2026-08-16). Data as JSON: /api/errors/03f1d1793cd915c7. Report an issue: GitHub.