What causes "bundled file range overlaps split footer" in quickwit-oss/quickwit?

Re-index or re-upload the split so its bundle metadata is regenerated consistently.

bundled file range overlaps split footer

Error message

bundled file range overlaps split footer

What it means

After locating the file's byte range and the split footer range, bundle storage requires the bundled file to lie entirely before the footer. If file_range.end exceeds footer_range.start, the file range overlaps the split footer (hotcache/bundle metadata) region, meaning the bundle metadata is inconsistent. Throwing prevents reading footer bytes as file content.

Source

Thrown at quickwit/quickwit-storage/src/bundle_storage.rs:134

            DEFAULT_SPLIT_TAIL_WINDOW_NUM_BYTES,
        )
        .await?;

        // Parse the bundle file ranges from the split bytes.
        let tail_start = split_len - split_bytes.len() as u64;
        let (file_ranges, _hotcache) =
            BundleFileRanges::open_from_split_bytes(split_bytes.clone())?;
        let file_range = file_ranges.get(&bundle_filepath).ok_or_else(|| {
            anyhow::anyhow!(
                "missing file `{}` in split bundle",
                bundle_filepath.display()
            )
        })?;
        ensure!(
            file_range.start <= file_range.end,
            "bundled file range starts after it ends"
        );
        ensure!(
            file_range.end <= footer_range.start,
            "bundled file range overlaps split footer"
        );

        // If the initial tail also contains the file, reuse it and complete in one GET (at least).
        // Otherwise, fetch the file with an additional GET.
        let file_bytes = if file_range.start >= tail_start {
            let relative_start = (file_range.start - tail_start) as usize;
            let relative_end = (file_range.end - tail_start) as usize;
            split_bytes.slice(relative_start..relative_end)
        } else {
            let relative_start = file_range.start as usize;
            let relative_end = file_range.end as usize;
            storage
                .get_slice(split_path, relative_start..relative_end)
                .await?
        };
        Ok((file_bytes, footer_range))

View on GitHub (pinned to a39730c5cd)

Solutions

Re-index or re-upload the split so its bundle metadata is regenerated consistently.
Verify the split's checksum against the metastore/manifest to confirm corruption before replacing it.
Check the writing Quickwit version for known bundle-footer bugs and upgrade if affected.

Defensive patterns

Strategy: try-catch

Try / catch

match bundle_storage.fetch_file_from_split(split, path).await {
    Ok(bytes) => bytes,
    Err(e) if e.to_string().contains("overlaps split footer") => {
        // treat split as corrupted; re-index or re-upload
    }
    Err(e) => return Err(e.into()),
}

Prevention

Verify split checksums when copying objects between buckets.
Never overwrite an existing split object in place; write a new object instead.
Run the same Quickwit version cluster-wide to avoid writer/reader layout drift.

When it happens

Trigger: Calling `fetch_file_from_split` on a split where the recorded file range crosses into the footer region — caused by corrupted or malformed bundle metadata in the split footer.

Common situations: Splits written with a buggy writer version; object-storage objects partially overwritten so the footer moved but file ranges didn't; manual manipulation of split files.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

AI-assisted analysis of quickwit-oss/quickwit@a39730c5cd (2026-09-08). Data as JSON: /api/errors/a60e492fb3919965. Report an issue: GitHub.