{"record":{"id":"905bf3d4a65755d4","repo":"neondatabase/neon","slug":"received-truncated-listobjectversions-response-for","errorCode":null,"errorMessage":"Received truncated ListObjectVersions response for prefix={prefix:?}","messagePattern":"Received truncated ListObjectVersions response for prefix=(.+?)","errorType":"exception","errorClass":"DownloadError","httpStatus":null,"severity":"error","filePath":"libs/remote_storage/src/s3_bucket.rs","lineNumber":511,"sourceCode":"                        key,\n                        last_modified,\n                        kind: crate::VersionKind::DeletionMarker,\n                    })\n                });\n            itertools::process_results(versions.chain(deletes), |n_vds| {\n                versions_and_deletes.extend(n_vds)\n            })\n            .map_err(DownloadError::Other)?;\n            fn none_if_empty(v: Option<String>) -> Option<String> {\n                v.filter(|v| !v.is_empty())\n            }\n            version_id_marker = none_if_empty(response.next_version_id_marker);\n            key_marker = none_if_empty(response.next_key_marker);\n            if version_id_marker.is_none() {\n                // The final response is not supposed to be truncated\n                if response.is_truncated.unwrap_or_default() {\n                    return Err(DownloadError::Other(anyhow::anyhow!(\n                        \"Received truncated ListObjectVersions response for prefix={prefix:?}\"\n                    )));\n                }\n                break;\n            }\n            if let Some(max_keys) = max_keys {\n                if versions_and_deletes.len() >= max_keys.get().try_into().unwrap() {\n                    return Err(DownloadError::Other(anyhow::anyhow!(\"too many versions\")));\n                }\n            }\n        }\n        Ok(VersionListing {\n            versions: versions_and_deletes,\n        })\n    }\n\n    pub fn bucket_name(&self) -> &str {\n        &self.bucket_name\n    }","sourceCodeStart":493,"sourceCodeEnd":529,"githubUrl":"https://github.com/neondatabase/neon/blob/8f60b04da47ffefe0e52bda2440134b42874eb75/libs/remote_storage/src/s3_bucket.rs#L493-L529","documentation":"While paginating ListObjectVersions, the loop stops when a response carries no NextVersionIdMarker; if that response still reports IsTruncated=true, the S3 pagination contract is broken and this error aborts the listing. Real S3 always emits continuation markers on truncated responses, so this points at a non-compliant S3-compatible endpoint.","triggerScenarios":"Calling list_versions or time_travel_recover against an emulator or gateway that truncates ListObjectVersions responses without setting next_version_id_marker/next_key_marker (or returns them empty).","commonSituations":"Testing against localstack-like emulators with partial ListObjectVersions support; S3 gateways that drop pagination params like max-keys or version-id-marker.","solutions":["Reproduce against real S3 or MinIO to confirm it is an endpoint bug","Upgrade the emulator/gateway to a version with correct ListObjectVersions pagination","File an issue with the endpoint vendor, including the prefix and page size used","As a workaround, narrow the prefix so the listing fits in fewer pages"],"exampleFix":"# before: CI uses an emulator with broken pagination\nNEON_REMOTE_STORAGE_S3_URL=http://emulator:5000/bucket\n# after\nNEON_REMOTE_STORAGE_S3_URL=http://minio:9000/bucket","handlingStrategy":"try-catch","validationCode":"use aws_sdk_s3::Client;\n\nasync fn pagination_is_compliant(client: &Client, bucket: &str) -> anyhow::Result<bool> {\n    let resp = client.list_object_versions().bucket(bucket).max_keys(1).send().await?;\n    let truncated = resp.is_truncated.unwrap_or(false);\n    let has_marker = resp.next_version_id_marker.as_deref().map(|m| !m.is_empty()).unwrap_or(false);\n    Ok(!truncated || has_marker)\n}","typeGuard":"fn is_truncated_listing_violation(err: &remote_storage::DownloadError) -> bool {\n    matches!(err, remote_storage::DownloadError::Other(e)\n        if e.to_string().contains(\"truncated ListObjectVersions\"))\n}","tryCatchPattern":"if let Err(DownloadError::Other(e)) = storage.list_versions(&prefix, mode, None, &cancel).await {\n    if e.to_string().contains(\"truncated ListObjectVersions\") {\n        // protocol violation: fail fast and flag the endpoint, do not retry\n    }\n}","preventionTips":["Run a pagination compliance probe before onboarding a new S3-compatible endpoint","Pin emulator versions known to implement continuation markers correctly"],"tags":["s3","rust","aws-sdk","pagination","emulator"],"backgroundTag":"s3-list-pagination-inconsistent","analyzedSha":"8f60b04da47ffefe0e52bda2440134b42874eb75","analyzedAt":"2026-08-16T23:39:28.135Z","schemaVersion":2},"datasetVersion":"2026-08-17T04:17:16.089Z"}