quickwit-oss/quickwit · error

split tail exceeds split length

Error message

split tail exceeds split length

What it means

read_split_tail validates that the requested number of tail bytes does not exceed the split's total length before computing the read range. The caller (fetch_split_tail) grows the tail window while looking for the footer start; if the window would reach before byte 0 of the split, the split is malformed or the size bookkeeping is wrong, so this invariant error is thrown.

Source

Thrown at quickwit/quickwit-storage/src/bundle_storage.rs:356

        }
    };

    // If the initial tail does not contain the entire footer, fetch the exact footer range now
    // that its boundaries are known.
    let footer_num_bytes = footer_range.end - footer_range.start;
    if (tail_bytes.len() as u64) < footer_num_bytes {
        tail_bytes = read_split_tail(storage, split_path, split_len, footer_num_bytes).await?;
    }
    Ok((tail_bytes, footer_range))
}

async fn read_split_tail(
    storage: &dyn Storage,
    split_path: &Path,
    split_len: u64,
    tail_num_bytes: u64,
) -> anyhow::Result<OwnedBytes> {
    ensure!(
        tail_num_bytes <= split_len,
        "split tail exceeds split length"
    );
    let start = (split_len - tail_num_bytes) as usize;
    let tail_bytes = storage
        .get_slice(split_path, start..split_len as usize)
        .await?;
    Ok(tail_bytes)
}

/// Removes the fixed split footer trailer when it is present.
pub fn strip_split_footer_trailer(split_slice: FileSlice) -> anyhow::Result<FileSlice> {
    if split_slice.len() < SPLIT_FOOTER_TRAILER_NUM_BYTES {
        return Ok(split_slice);
    }
    let (split_slice_without_trailer, trailer) = split_slice
        .clone()
        .split_from_end(SPLIT_FOOTER_TRAILER_NUM_BYTES);

View on GitHub (pinned to a39730c5cd)

Solutions

1. Verify the split was produced by a compatible Quickwit version; re-index or migrate data written by an incompatible format.
2. Check that split_len passed to fetch_split_tail equals the real object size in storage; refresh it via storage.file_num_bytes.
3. Download the split and inspect its trailer/footer bytes to confirm corruption; if corrupted, restore from backup or delete and re-index.
4. Ensure the file being read is actually a Quickwit split bundle, not a raw tantivy file or other artifact.

Example fix

// before
let tail_num_bytes = std::cmp::max(tail_num_bytes * 2, MIN_TAIL);
let tail = read_split_tail(storage, &split_path, split_len, tail_num_bytes).await?;
// after
let tail_num_bytes = std::cmp::max(tail_num_bytes * 2, MIN_TAIL);
if tail_num_bytes > split_len {
    return Err(anyhow::anyhow!("footer of split {} unreadable/corrupt", split_path));
}
let tail = read_split_tail(storage, &split_path, split_len, tail_num_bytes).await?;

if tail_num_bytes > split_len {
    anyhow::bail!("tail window {} exceeds split length {}", tail_num_bytes, split_len);
}

match read_split_tail(storage, &split_path, split_len, tail_num_bytes).await {
    Err(e) if e.to_string().contains("split tail exceeds split length") => {
        anyhow::bail!("corrupt or incompatible split footer in {}", split_path);
    }
    other => other,
}

When it happens

Trigger: fetch_split_tail exponentially grows its tail window across calls to read_split_tail; this fires when the grown tail_num_bytes surpasses split_len, i.e. a split whose footer is corrupt/unparseable or whose advertised length does not match the actual footer layout.

Common situations: Splits written by an incompatible or older Quickwit version whose footer format differs; a corrupted footer that fails to parse even at full-tail size; manual edits or reserialization of split files; reading a file that is not a Quickwit split at all.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• failed to locate split footer after reading bundle metadata
• split is too short to contain a footer
• split tail is longer than the split itself
• split footer starts after its trailer
• no parent directory for {full_path:?}

AI-assisted analysis of quickwit-oss/quickwit@a39730c5cd (2026-09-08). Data as JSON: /api/errors/5fd94c7b4b7854b6. Report an issue: GitHub.