quickwit-oss/quickwit · error

split tail is longer than the split itself

Error message

split tail is longer than the split itself

What it means

locate_split_footer_range_in_tail reads only the tail of the split and must therefore satisfy tail_len <= split_len. If the fetched tail is longer than the whole split, the split length and tail are inconsistent — usually a corrupted split, a wrong reported file length from storage, or a race where the split shrank. The check catches this before parsing the trailer.

Source

Thrown at quickwit/quickwit-storage/src/bundle_storage.rs:249

fn locate_split_footer_range_from_metadata_len(
    split_len: u64,
    bundle_metadata_len_start: u64,
    bundle_metadata_len_bytes: &[u8],
) -> anyhow::Result<Range<u64>> {
    let bundle_metadata_len = u32::from_le_bytes(bundle_metadata_len_bytes.try_into()?) as u64;
    let footer_start = bundle_metadata_len_start
        .checked_sub(bundle_metadata_len)
        .context("split footer exceeds split length")?;
    Ok(footer_start..split_len)
}

fn locate_split_footer_range_in_tail(
    split_len: u64,
    tail_bytes: &[u8],
) -> anyhow::Result<FooterLocation> {
    // Legacy split layout:
    // [body][bundle metadata][metadata len][hotcache][hotcache len]
    ensure!(
        tail_bytes.len() as u64 <= split_len,
        "split tail is longer than the split itself"
    );
    let trailer_start = tail_bytes
        .len()
        .checked_sub(SPLIT_FOOTER_TRAILER_NUM_BYTES)
        .context("split tail is too short to contain a footer trailer")?;
    if let Some(footer_start) = deserialize_split_footer_trailer(&tail_bytes[trailer_start..])? {
        ensure!(
            footer_start <= split_len - SPLIT_FOOTER_TRAILER_NUM_BYTES as u64,
            "split footer starts after its trailer"
        );
        return Ok(FooterLocation::Located(footer_start..split_len));
    }

    let hotcache_len =
        u32::from_le_bytes(tail_bytes[tail_bytes.len() - HOTCACHE_LEN_NUM_BYTES..].try_into()?)
            as u64;

View on GitHub (pinned to a39730c5cd)

Solutions

1. Re-upload or re-index the split to make the stored object and its recorded length consistent.
2. Compare the object's actual size in storage with split_len in the metastore; fix the metastore record if it is stale.
3. Check for concurrent writers/overwrites of the same split file and eliminate them.

let actual_len = storage.object_metadata(&path).await?.content_length;
if tail_request_len > actual_len { /* refresh metastore split_len or re-upload before reading */ }

match locate_split_footer_range(split, &storage).await {
    Ok(f) => f,
    Err(e) if e.to_string().contains("tail is longer than the split") => {
        // refresh object metadata / retry once; if persistent, re-upload the split
    }
    Err(e) => return Err(e.into()),
}

When it happens

Trigger: Calling `locate_split_footer_range` / `fetch_split_tail` where the bytes returned for the tail exceed the split's length (split_len from storage metadata is smaller than the actual tail read, or the object shrank concurrently).

Common situations: Object storage returning stale/inconsistent length metadata; splits overwritten or truncated by another process; mismatched metastore size vs. actual object after a failed upload overwrite.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• failed to locate split footer after reading bundle metadata
• split is too short to contain a footer
• split tail exceeds split length
• split footer starts after its trailer
• bundled file range starts after it ends

AI-assisted analysis of quickwit-oss/quickwit@a39730c5cd (2026-09-08). Data as JSON: /api/errors/91fff1ce21054917. Report an issue: GitHub.