{"record":{"id":"91fff1ce21054917","repo":"quickwit-oss/quickwit","slug":"split-tail-is-longer-than-the-split-itself","errorCode":null,"errorMessage":"split tail is longer than the split itself","messagePattern":"split tail is longer than the split itself","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"quickwit/quickwit-storage/src/bundle_storage.rs","lineNumber":249,"sourceCode":"fn locate_split_footer_range_from_metadata_len(\n    split_len: u64,\n    bundle_metadata_len_start: u64,\n    bundle_metadata_len_bytes: &[u8],\n) -> anyhow::Result<Range<u64>> {\n    let bundle_metadata_len = u32::from_le_bytes(bundle_metadata_len_bytes.try_into()?) as u64;\n    let footer_start = bundle_metadata_len_start\n        .checked_sub(bundle_metadata_len)\n        .context(\"split footer exceeds split length\")?;\n    Ok(footer_start..split_len)\n}\n\nfn locate_split_footer_range_in_tail(\n    split_len: u64,\n    tail_bytes: &[u8],\n) -> anyhow::Result<FooterLocation> {\n    // Legacy split layout:\n    // [body][bundle metadata][metadata len][hotcache][hotcache len]\n    ensure!(\n        tail_bytes.len() as u64 <= split_len,\n        \"split tail is longer than the split itself\"\n    );\n    let trailer_start = tail_bytes\n        .len()\n        .checked_sub(SPLIT_FOOTER_TRAILER_NUM_BYTES)\n        .context(\"split tail is too short to contain a footer trailer\")?;\n    if let Some(footer_start) = deserialize_split_footer_trailer(&tail_bytes[trailer_start..])? {\n        ensure!(\n            footer_start <= split_len - SPLIT_FOOTER_TRAILER_NUM_BYTES as u64,\n            \"split footer starts after its trailer\"\n        );\n        return Ok(FooterLocation::Located(footer_start..split_len));\n    }\n\n    let hotcache_len =\n        u32::from_le_bytes(tail_bytes[tail_bytes.len() - HOTCACHE_LEN_NUM_BYTES..].try_into()?)\n            as u64;","sourceCodeStart":231,"sourceCodeEnd":267,"githubUrl":"https://github.com/quickwit-oss/quickwit/blob/a39730c5cdcd1a4fe798403737ae293999ea21f8/quickwit/quickwit-storage/src/bundle_storage.rs#L231-L267","documentation":"locate_split_footer_range_in_tail reads only the tail of the split and must therefore satisfy tail_len <= split_len. If the fetched tail is longer than the whole split, the split length and tail are inconsistent — usually a corrupted split, a wrong reported file length from storage, or a race where the split shrank. The check catches this before parsing the trailer.","triggerScenarios":"Calling `locate_split_footer_range` / `fetch_split_tail` where the bytes returned for the tail exceed the split's length (split_len from storage metadata is smaller than the actual tail read, or the object shrank concurrently).","commonSituations":"Object storage returning stale/inconsistent length metadata; splits overwritten or truncated by another process; mismatched metastore size vs. actual object after a failed upload overwrite.","solutions":["Re-upload or re-index the split to make the stored object and its recorded length consistent.","Compare the object's actual size in storage with split_len in the metastore; fix the metastore record if it is stale.","Check for concurrent writers/overwrites of the same split file and eliminate them."],"exampleFix":null,"handlingStrategy":"retry","validationCode":"let actual_len = storage.object_metadata(&path).await?.content_length;\nif tail_request_len > actual_len { /* refresh metastore split_len or re-upload before reading */ }","typeGuard":null,"tryCatchPattern":"match locate_split_footer_range(split, &storage).await {\n    Ok(f) => f,\n    Err(e) if e.to_string().contains(\"tail is longer than the split\") => {\n        // refresh object metadata / retry once; if persistent, re-upload the split\n    }\n    Err(e) => return Err(e.into()),\n}","preventionTips":["Ensure metastore split_len matches the object's actual size after uploads (finalize-then-publish).","Prevent concurrent overwrites of split objects.","Retry transient reads with fresh metadata before concluding corruption."],"tags":["storage","split-footer","corruption"],"backgroundTag":"internal-invariant-violation","analyzedSha":"a39730c5cdcd1a4fe798403737ae293999ea21f8","analyzedAt":"2026-09-08T13:19:37.784Z","contentChangedAt":"2026-09-08T13:19:37.784Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}