neondatabase/neon · error

too many shards: {n}

Error message

too many shards: {n}

What it means

ShardSpec::new in pageserver/client_grpc builds a tenant shard specification from a HashMap of ShardIndex to URL. The shard count is stored in a u8 (ShardCount), and 0 is reserved for unsharded tenants, so at most 255 shard URLs are accepted. A map with more than 255 entries is rejected with this error before any other validation runs.

Source

Thrown at pageserver/client_grpc/src/client.rs:397

    count: ShardCount,
    /// The stripe size for these shards.
    ///
    /// INVARIANT: None for unsharded tenants, Some for sharded.
    stripe_size: Option<ShardStripeSize>,
}

impl ShardSpec {
    /// Creates a new shard spec with the given URLs and stripe size. All shards must be given.
    /// The stripe size must be Some for sharded tenants, or None for unsharded tenants.
    pub fn new(
        urls: HashMap<ShardIndex, String>,
        stripe_size: Option<ShardStripeSize>,
    ) -> anyhow::Result<Self> {
        // Compute the shard count.
        let count = match urls.len() {
            0 => return Err(anyhow!("no shards provided")),
            1 => ShardCount::new(0), // NB: unsharded tenants use 0, like `ShardIndex::unsharded()`
            n if n > u8::MAX as usize => return Err(anyhow!("too many shards: {n}")),
            n => ShardCount::new(n as u8),
        };

        // Validate the stripe size.
        if stripe_size.is_none() && !count.is_unsharded() {
            return Err(anyhow!("stripe size must be given for sharded tenants"));
        }
        if stripe_size.is_some() && count.is_unsharded() {
            return Err(anyhow!("stripe size can't be given for unsharded tenants"));
        }

        // Validate the shard spec.
        for (shard_id, url) in &urls {
            // The shard index must match the computed shard count, even for unsharded tenants.
            if shard_id.shard_count != count {
                return Err(anyhow!("invalid shard index {shard_id}, expected {count}"));
            }
            // The shard index' number and count must be consistent.

View on GitHub (pinned to 8f60b04da4)

Solutions

  1. Reduce the shard set to 255 or fewer URLs; ShardCount is u8-backed with 0 reserved for unsharded, so 255 is a hard maximum.
  2. Inspect the code that builds the HashMap for duplicated entries or generation loops that run twice (e.g. 1..=N appended two times).
  3. If the tenant genuinely needs more than 255 shards, split the tenant or raise the limit upstream in ShardCount; never truncate the map to sneak past the check.

Example fix

// before
let spec = ShardSpec::new(all_shard_urls, Some(stripe))?; // 300 entries -> "too many shards: 300"

// after
assert!(all_shard_urls.len() <= u8::MAX as usize, "max 255 shards");
let spec = ShardSpec::new(all_shard_urls, Some(stripe))?;
Defensive patterns

Strategy: validation

Validate before calling

// before calling ShardSpec::new
const MAX_SHARDS: usize = u8::MAX as usize; // 0 reserved for unsharded
anyhow::ensure!(
    (1..=MAX_SHARDS).contains(&urls.len()),
    "shard map has {} entries; 1..=255 required",
    urls.len()
);

Type guard

fn shard_count_representable(urls: &HashMap<ShardIndex, String>) -> bool {
    (1..=u8::MAX as usize).contains(&urls.len())
}

Try / catch

match ShardSpec::new(urls, stripe_size) {
    Ok(spec) => spec,
    Err(e) if e.to_string().starts_with("too many shards") => {
        // reduce the shard set, then rebuild the spec with regenerated indices
        return Err(e.context("shard set exceeds u8 ShardCount; split the tenant"));
    }
    Err(e) => return Err(e),
}

Prevention

When it happens

Trigger: Calling ShardSpec::new(urls, stripe_size), or any control-plane spec parser that feeds it, with urls.len() greater than 255 (u8::MAX). The count check runs first, so stripe size and shard index validation never execute.

Common situations: A test harness or control plane generates N shards from a config value without clamping N; a script duplicates shard URLs into an oversized map; an operator tries to scale a single tenant beyond the current 255-shard ceiling.

Related errors

AI-assisted analysis of neondatabase/neon@8f60b04da4 (2026-08-16). Data as JSON: /api/errors/7801269c0fe14bb3. Report an issue: GitHub.