neondatabase/neon · warning · DownloadError

too many versions

Error message

too many versions

What it means

list_versions_with_permit accepts an optional max_keys limit; after each page it checks whether accumulated versions plus delete markers already meet or exceed that limit and fails with 'too many versions' rather than silently truncating. In practice this surfaces through time_travel_recover, which forwards its complexity_limit as max_keys.

Source

Thrown at libs/remote_storage/src/s3_bucket.rs:518

            })
            .map_err(DownloadError::Other)?;
            fn none_if_empty(v: Option<String>) -> Option<String> {
                v.filter(|v| !v.is_empty())
            }
            version_id_marker = none_if_empty(response.next_version_id_marker);
            key_marker = none_if_empty(response.next_key_marker);
            if version_id_marker.is_none() {
                // The final response is not supposed to be truncated
                if response.is_truncated.unwrap_or_default() {
                    return Err(DownloadError::Other(anyhow::anyhow!(
                        "Received truncated ListObjectVersions response for prefix={prefix:?}"
                    )));
                }
                break;
            }
            if let Some(max_keys) = max_keys {
                if versions_and_deletes.len() >= max_keys.get().try_into().unwrap() {
                    return Err(DownloadError::Other(anyhow::anyhow!("too many versions")));
                }
            }
        }
        Ok(VersionListing {
            versions: versions_and_deletes,
        })
    }

    pub fn bucket_name(&self) -> &str {
        &self.bucket_name
    }
}

pin_project_lite::pin_project! {
    struct ByteStreamAsStream {
        #[pin]
        inner: aws_smithy_types::byte_stream::ByteStream
    }

View on GitHub (pinned to 8f60b04da4)

Solutions

1. Increase the max_keys/complexity_limit passed to the listing or time-travel API
2. Reduce version count under the prefix (lifecycle rules, retention cleanup)
3. If a complete enumeration is intended, pass None as the limit

Example fix

// before
let limit = NonZeroU32::new(1000);
storage.time_travel_recover(&prefix, ts, done_if_after, &cancel, limit).await?;
// after: raise the complexity limit to cover actual version churn
let limit = NonZeroU32::new(100_000);
storage.time_travel_recover(&prefix, ts, done_if_after, &cancel, limit).await?;

use std::num::NonZeroU32;

// pick a limit comfortably above measured version churn per prefix,
// or None when a complete enumeration is required anyway
fn pick_limit(measured_versions: u32) -> Option<NonZeroU32> {
    NonZeroU32::new(measured_versions.saturating_mul(2).max(1000))
}

fn is_too_many_versions(err: &remote_storage::DownloadError) -> bool {
    matches!(err, remote_storage::DownloadError::Other(e)
        if e.to_string().contains("too many versions"))
}

match storage.list_versions(&prefix, mode, limit, &cancel).await {
    Err(DownloadError::Other(e)) if e.to_string().contains("too many versions") => {
        // decide: raise the limit and retry, or shed this tenant/prefix
    }
    other => other?,
}

When it happens

Trigger: Calling time_travel_recover (or list_versions) with a max_keys/complexity_limit smaller than the number of object versions plus delete markers under the prefix, e.g. a timeline prefix with heavy layer upload/delete churn.

Common situations: Time-travel retention runs on busy tenants whose version churn per prefix exceeds the configured complexity limit; limits tuned before branching-heavy workloads increased version counts.

Related errors

• Received truncated ListObjectVersions response for prefix={p
• Received ListVersions response for key={key} with version_id
• Read back file doesn't match original
• max keys reached
• Missing ETag header

AI-assisted analysis of neondatabase/neon@8f60b04da4 (2026-08-16). Data as JSON: /api/errors/291115859c1fdf1b. Report an issue: GitHub.