quickwit-oss/quickwit · error

split is too short to contain a footer

Error message

split is too short to contain a footer

What it means

locate_split_footer_range refuses to read a footer when the split file's total length is smaller than the fixed SPLIT_FOOTER_TRAILER_NUM_BYTES trailer. Every Quickwit split ends with a fixed-size trailer holding the footer metadata, so a split shorter than that trailer cannot be a valid split. This is a corruption/truncation guard raised via anyhow::ensure! before any storage read.

Solutions

1. Check the actual size of the split object in the object store and compare it with the expected size recorded at publish time; if smaller, the upload was truncated — delete the split and re-index the affected data.
2. Verify the path passed to read_split_footer/open_from_storage really points to a split file, not a hotcache or metadata file.
3. Re-upload or restore the split from a known-good copy and confirm the checksum matches before retrying.
4. If this reproduces on freshly indexed splits, inspect the uploader/packager stage for bugs writing incomplete bundles.

Example fix

// before: blindly open whatever the metastore points at
let footer = read_split_footer(storage, &split_path, len).await?;
// after: guard on size first
if len < SPLIT_FOOTER_TRAILER_NUM_BYTES as u64 {
    return Err(anyhow::anyhow!("split {} truncated ({} bytes), re-index required", split_path, len));
}
let footer = read_split_footer(storage, &split_path, len).await?;

let split_len = storage.file_num_bytes(&split_path).await?;
if split_len < quickwit_storage::SPLIT_FOOTER_TRAILER_NUM_BYTES as u64 {
    anyhow::bail!("split {} is truncated ({} bytes)", split_path, split_len);
}

fn split_size_plausible(split_len: u64) -> bool {
    split_len >= quickwit_storage::SPLIT_FOOTER_TRAILER_NUM_BYTES as u64
}

When it happens

Trigger: Calling read_split_footer or BundleStorage::open_from_storage on a split whose byte length (as reported by storage.file_num_bytes or the metastore) is less than SPLIT_FOOTER_TRAILER_NUM_BYTES. Typically the split upload was truncated, partially copied, or the wrong object was uploaded under the split path.

Common situations: Interrupted split uploads to S3/GCS leaving partial objects; manual copy/restore of split files that got truncated; a hotcache or staging file accidentally passed instead of the real split; metastore pointing at an object overwritten by a zero-byte or partial upload.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• failed to locate split footer after reading bundle metadata…
• split footer starts after its trailer
• split tail exceeds split length
• split tail is longer than the split itself
• bundled file range overlaps split footer

AI-assisted analysis of quickwit-oss/quickwit@a39730c5cd (2026-09-08). Data as JSON: /api/errors/92aaa77a44fd88c9. Report an issue: GitHub.

Appendix: source

Thrown at quickwit/quickwit-storage/src/bundle_storage.rs:203

    let version = reader.get_u32_le();

    if reader != SPLIT_FOOTER_TRAILER_MAGIC {
        return Ok(None);
    }
    ensure!(
        version == SPLIT_FOOTER_TRAILER_VERSION,
        "unsupported split footer trailer version {version}"
    );
    Ok(Some(footer_start_inclusive))
}

/// Locates a split footer range using its fixed trailer, with support for legacy split layouts.
pub async fn locate_split_footer_range(
    storage: &dyn Storage,
    split_path: &Path,
    split_len: u64,
) -> anyhow::Result<Range<u64>> {
    ensure!(
        split_len >= SPLIT_FOOTER_TRAILER_NUM_BYTES as u64,
        "split is too short to contain a footer"
    );
    let end = split_len as usize;
    let start = end - SPLIT_FOOTER_TRAILER_NUM_BYTES;
    let tail_bytes = storage.get_slice(split_path, start..end).await?;
    match locate_split_footer_range_in_tail(split_len, &tail_bytes)? {
        FooterLocation::Located(footer_range) => Ok(footer_range),
        FooterLocation::ReadBundleMetadataLen(bundle_metadata_len_range) => {
            let start = bundle_metadata_len_range.start as usize;
            let end = bundle_metadata_len_range.end as usize;
            let bundle_metadata_len_bytes = storage.get_slice(split_path, start..end).await?;
            locate_split_footer_range_from_metadata_len(
                split_len,
                bundle_metadata_len_range.start,
                bundle_metadata_len_bytes.as_slice(),
            )
        }

View on GitHub (pinned to a39730c5cd)