{"record":{"id":"92aaa77a44fd88c9","repo":"quickwit-oss/quickwit","slug":"split-is-too-short-to-contain-a-footer","errorCode":null,"errorMessage":"split is too short to contain a footer","messagePattern":"split is too short to contain a footer","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"quickwit/quickwit-storage/src/bundle_storage.rs","lineNumber":203,"sourceCode":"    let version = reader.get_u32_le();\n\n    if reader != SPLIT_FOOTER_TRAILER_MAGIC {\n        return Ok(None);\n    }\n    ensure!(\n        version == SPLIT_FOOTER_TRAILER_VERSION,\n        \"unsupported split footer trailer version {version}\"\n    );\n    Ok(Some(footer_start_inclusive))\n}\n\n/// Locates a split footer range using its fixed trailer, with support for legacy split layouts.\npub async fn locate_split_footer_range(\n    storage: &dyn Storage,\n    split_path: &Path,\n    split_len: u64,\n) -> anyhow::Result<Range<u64>> {\n    ensure!(\n        split_len >= SPLIT_FOOTER_TRAILER_NUM_BYTES as u64,\n        \"split is too short to contain a footer\"\n    );\n    let end = split_len as usize;\n    let start = end - SPLIT_FOOTER_TRAILER_NUM_BYTES;\n    let tail_bytes = storage.get_slice(split_path, start..end).await?;\n    match locate_split_footer_range_in_tail(split_len, &tail_bytes)? {\n        FooterLocation::Located(footer_range) => Ok(footer_range),\n        FooterLocation::ReadBundleMetadataLen(bundle_metadata_len_range) => {\n            let start = bundle_metadata_len_range.start as usize;\n            let end = bundle_metadata_len_range.end as usize;\n            let bundle_metadata_len_bytes = storage.get_slice(split_path, start..end).await?;\n            locate_split_footer_range_from_metadata_len(\n                split_len,\n                bundle_metadata_len_range.start,\n                bundle_metadata_len_bytes.as_slice(),\n            )\n        }","sourceCodeStart":185,"sourceCodeEnd":221,"githubUrl":"https://github.com/quickwit-oss/quickwit/blob/a39730c5cdcd1a4fe798403737ae293999ea21f8/quickwit/quickwit-storage/src/bundle_storage.rs#L185-L221","documentation":"locate_split_footer_range refuses to read a footer when the split file's total length is smaller than the fixed SPLIT_FOOTER_TRAILER_NUM_BYTES trailer. Every Quickwit split ends with a fixed-size trailer holding the footer metadata, so a split shorter than that trailer cannot be a valid split. This is a corruption/truncation guard raised via anyhow::ensure! before any storage read.","triggerScenarios":"Calling read_split_footer or BundleStorage::open_from_storage on a split whose byte length (as reported by storage.file_num_bytes or the metastore) is less than SPLIT_FOOTER_TRAILER_NUM_BYTES. Typically the split upload was truncated, partially copied, or the wrong object was uploaded under the split path.","commonSituations":"Interrupted split uploads to S3/GCS leaving partial objects; manual copy/restore of split files that got truncated; a hotcache or staging file accidentally passed instead of the real split; metastore pointing at an object overwritten by a zero-byte or partial upload.","solutions":["Check the actual size of the split object in the object store and compare it with the expected size recorded at publish time; if smaller, the upload was truncated — delete the split and re-index the affected data.","Verify the path passed to read_split_footer/open_from_storage really points to a split file, not a hotcache or metadata file.","Re-upload or restore the split from a known-good copy and confirm the checksum matches before retrying.","If this reproduces on freshly indexed splits, inspect the uploader/packager stage for bugs writing incomplete bundles."],"exampleFix":"// before: blindly open whatever the metastore points at\nlet footer = read_split_footer(storage, &split_path, len).await?;\n// after: guard on size first\nif len < SPLIT_FOOTER_TRAILER_NUM_BYTES as u64 {\n    return Err(anyhow::anyhow!(\"split {} truncated ({} bytes), re-index required\", split_path, len));\n}\nlet footer = read_split_footer(storage, &split_path, len).await?;","handlingStrategy":"validation","validationCode":"let split_len = storage.file_num_bytes(&split_path).await?;\nif split_len < quickwit_storage::SPLIT_FOOTER_TRAILER_NUM_BYTES as u64 {\n    anyhow::bail!(\"split {} is truncated ({} bytes)\", split_path, split_len);\n}","typeGuard":"fn split_size_plausible(split_len: u64) -> bool {\n    split_len >= quickwit_storage::SPLIT_FOOTER_TRAILER_NUM_BYTES as u64\n}","tryCatchPattern":null,"preventionTips":["Verify uploaded split sizes/checksums after upload completes before publishing to the metastore.","Never point the metastore at objects you did not fully upload.","Monitor object-store Content-Length vs expected split size as an operational alert."],"tags":["storage","corruption","split-footer","truncation"],"backgroundTag":"file-read-failed","analyzedSha":"a39730c5cdcd1a4fe798403737ae293999ea21f8","analyzedAt":"2026-09-08T13:19:37.784Z","contentChangedAt":"2026-09-08T13:19:37.784Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}