neondatabase/neon · error

can't reduce shard count from {} to {}

Error message

can't reduce shard count from {} to {}

What it means

`ShardedPageserverClient::update_shards` (gRPC pageserver client) only supports growing the shard set, because Neon sharding uses splits — existing key→shard mappings stay valid when count increases but cannot be retracted. Calling it with a `ShardSpec` whose count is lower than the currently installed one returns this error without changing state.

Source

Thrown at pageserver/client_grpc/src/client.rs:134

            compression,
            shards: ArcSwap::new(Arc::new(shards)),
        })
    }

    /// Updates the shards from the given shard spec. In-flight requests will complete using the
    /// existing shards, but may retry with the new shards if they fail.
    ///
    /// TODO: verify that in-flight requests are allowed to complete, and that the old pools are
    /// properly spun down and dropped afterwards.
    pub fn update_shards(&self, shard_spec: ShardSpec) -> anyhow::Result<()> {
        // Validate the shard spec. We should really use `ArcSwap::rcu` for this, to avoid races
        // with concurrent updates, but that involves creating a new `Shards` on every attempt,
        // which spins up a bunch of Tokio tasks and such. These should already be checked elsewhere
        // in the stack, and if they're violated then we already have problems elsewhere, so a
        // best-effort but possibly-racy check is okay here.
        let old = self.shards.load_full();
        if shard_spec.count < old.count {
            return Err(anyhow!(
                "can't reduce shard count from {} to {}",
                old.count,
                shard_spec.count
            ));
        }
        if !old.count.is_unsharded() && shard_spec.stripe_size != old.stripe_size {
            return Err(anyhow!(
                "can't change stripe size from {} to {}",
                old.stripe_size.expect("always Some when sharded"),
                shard_spec.stripe_size.expect("always Some when sharded")
            ));
        }

        let shards = Shards::new(
            self.tenant_id,
            self.timeline_id,
            shard_spec,
            self.auth_token.clone(),

View on GitHub (pinned to 8f60b04da4)

Solutions

1. Don't reduce shard counts — Neon shards only split; construct the client fresh with the smaller spec instead of updating in place
2. Fix the caller (controller/config) to always pass the current-or-larger shard count
3. Check for stale shard-map caches feeding outdated ShardSpecs into update_shards
4. Verify the spec's count derivation (urls.len()) matches the intended topology

Example fix

// before: attempts to shrink from 8 shards to 4
client.update_shards(ShardSpec::new(four_shard_urls, stripe)?)?; // can't reduce shard count

// after: only grow, or rebuild the client for a different topology
if new_spec.count >= current.count {
    client.update_shards(new_spec)?;
} else {
    let client = ShardedPageserverClient::new(tenant_id, timeline_id, new_spec, ...)?;
}

let old = client.shards();
if shard_spec.count < old.count {
    anyhow::bail!(
        "shard count can only grow (current {}, requested {}); rebuild the client instead",
        old.count, shard_spec.count
    );
}
client.update_shards(shard_spec)?;

When it happens

Trigger: Invoking `update_shards(shard_spec)` where `shard_spec.count <` the client's current shard count — e.g. feeding a stale/smaller shard map from the controller after shards were already split.

Common situations: Controller reconciliation bugs replaying an old shard map; mixing up shard-count semantics (0 = unsharded vs. actual counts) when building the spec; tests attempting to 'unsplit' shards.

Related errors

• can't change stripe size from {} to {}
• no shards provided
• only unsharded tenants are supported at this time: {}
• too many shards: {n}
• stripe size must be given for sharded tenants

AI-assisted analysis of neondatabase/neon@8f60b04da4 (2026-08-16). Data as JSON: /api/errors/4f7494fba39a2c53. Report an issue: GitHub.