{"record":{"id":"5fd94c7b4b7854b6","repo":"quickwit-oss/quickwit","slug":"split-tail-exceeds-split-length","errorCode":null,"errorMessage":"split tail exceeds split length","messagePattern":"split tail exceeds split length","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"quickwit/quickwit-storage/src/bundle_storage.rs","lineNumber":356,"sourceCode":"        }\n    };\n\n    // If the initial tail does not contain the entire footer, fetch the exact footer range now\n    // that its boundaries are known.\n    let footer_num_bytes = footer_range.end - footer_range.start;\n    if (tail_bytes.len() as u64) < footer_num_bytes {\n        tail_bytes = read_split_tail(storage, split_path, split_len, footer_num_bytes).await?;\n    }\n    Ok((tail_bytes, footer_range))\n}\n\nasync fn read_split_tail(\n    storage: &dyn Storage,\n    split_path: &Path,\n    split_len: u64,\n    tail_num_bytes: u64,\n) -> anyhow::Result<OwnedBytes> {\n    ensure!(\n        tail_num_bytes <= split_len,\n        \"split tail exceeds split length\"\n    );\n    let start = (split_len - tail_num_bytes) as usize;\n    let tail_bytes = storage\n        .get_slice(split_path, start..split_len as usize)\n        .await?;\n    Ok(tail_bytes)\n}\n\n/// Removes the fixed split footer trailer when it is present.\npub fn strip_split_footer_trailer(split_slice: FileSlice) -> anyhow::Result<FileSlice> {\n    if split_slice.len() < SPLIT_FOOTER_TRAILER_NUM_BYTES {\n        return Ok(split_slice);\n    }\n    let (split_slice_without_trailer, trailer) = split_slice\n        .clone()\n        .split_from_end(SPLIT_FOOTER_TRAILER_NUM_BYTES);","sourceCodeStart":338,"sourceCodeEnd":374,"githubUrl":"https://github.com/quickwit-oss/quickwit/blob/a39730c5cdcd1a4fe798403737ae293999ea21f8/quickwit/quickwit-storage/src/bundle_storage.rs#L338-L374","documentation":"read_split_tail validates that the requested number of tail bytes does not exceed the split's total length before computing the read range. The caller (fetch_split_tail) grows the tail window while looking for the footer start; if the window would reach before byte 0 of the split, the split is malformed or the size bookkeeping is wrong, so this invariant error is thrown.","triggerScenarios":"fetch_split_tail exponentially grows its tail window across calls to read_split_tail; this fires when the grown tail_num_bytes surpasses split_len, i.e. a split whose footer is corrupt/unparseable or whose advertised length does not match the actual footer layout.","commonSituations":"Splits written by an incompatible or older Quickwit version whose footer format differs; a corrupted footer that fails to parse even at full-tail size; manual edits or reserialization of split files; reading a file that is not a Quickwit split at all.","solutions":["Verify the split was produced by a compatible Quickwit version; re-index or migrate data written by an incompatible format.","Check that split_len passed to fetch_split_tail equals the real object size in storage; refresh it via storage.file_num_bytes.","Download the split and inspect its trailer/footer bytes to confirm corruption; if corrupted, restore from backup or delete and re-index.","Ensure the file being read is actually a Quickwit split bundle, not a raw tantivy file or other artifact."],"exampleFix":"// before\nlet tail_num_bytes = std::cmp::max(tail_num_bytes * 2, MIN_TAIL);\nlet tail = read_split_tail(storage, &split_path, split_len, tail_num_bytes).await?;\n// after\nlet tail_num_bytes = std::cmp::max(tail_num_bytes * 2, MIN_TAIL);\nif tail_num_bytes > split_len {\n    return Err(anyhow::anyhow!(\"footer of split {} unreadable/corrupt\", split_path));\n}\nlet tail = read_split_tail(storage, &split_path, split_len, tail_num_bytes).await?;","handlingStrategy":"try-catch","validationCode":"if tail_num_bytes > split_len {\n    anyhow::bail!(\"tail window {} exceeds split length {}\", tail_num_bytes, split_len);\n}","typeGuard":null,"tryCatchPattern":"match read_split_tail(storage, &split_path, split_len, tail_num_bytes).await {\n    Err(e) if e.to_string().contains(\"split tail exceeds split length\") => {\n        anyhow::bail!(\"corrupt or incompatible split footer in {}\", split_path);\n    }\n    other => other,\n}","preventionTips":["Pin indexer and searcher to compatible Quickwit versions so footer formats match.","Never hand-edit or re-serialize split files.","Detect non-split files early by checking the trailer magic before footer parsing."],"tags":["storage","corruption","invariant-violation","split-footer"],"backgroundTag":"internal-invariant-violation","analyzedSha":"a39730c5cdcd1a4fe798403737ae293999ea21f8","analyzedAt":"2026-09-08T13:19:37.784Z","contentChangedAt":"2026-09-08T13:19:37.784Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}