quickwit-oss/quickwit · error

split is too short to contain a footer

Error message

split is too short to contain a footer

What it means

locate_split_footer_range refuses to read a footer when the split file's total length is smaller than the fixed SPLIT_FOOTER_TRAILER_NUM_BYTES trailer. Every Quickwit split ends with a fixed-size trailer holding the footer metadata, so a split shorter than that trailer cannot be a valid split. This is a corruption/truncation guard raised via anyhow::ensure! before any storage read.

Source

Thrown at quickwit/quickwit-storage/src/bundle_storage.rs:203

    let version = reader.get_u32_le();

    if reader != SPLIT_FOOTER_TRAILER_MAGIC {
        return Ok(None);
    }
    ensure!(
        version == SPLIT_FOOTER_TRAILER_VERSION,
        "unsupported split footer trailer version {version}"
    );
    Ok(Some(footer_start_inclusive))
}

/// Locates a split footer range using its fixed trailer, with support for legacy split layouts.
pub async fn locate_split_footer_range(
    storage: &dyn Storage,
    split_path: &Path,
    split_len: u64,
) -> anyhow::Result<Range<u64>> {
    ensure!(
        split_len >= SPLIT_FOOTER_TRAILER_NUM_BYTES as u64,
        "split is too short to contain a footer"
    );
    let end = split_len as usize;
    let start = end - SPLIT_FOOTER_TRAILER_NUM_BYTES;
    let tail_bytes = storage.get_slice(split_path, start..end).await?;
    match locate_split_footer_range_in_tail(split_len, &tail_bytes)? {
        FooterLocation::Located(footer_range) => Ok(footer_range),
        FooterLocation::ReadBundleMetadataLen(bundle_metadata_len_range) => {
            let start = bundle_metadata_len_range.start as usize;
            let end = bundle_metadata_len_range.end as usize;
            let bundle_metadata_len_bytes = storage.get_slice(split_path, start..end).await?;
            locate_split_footer_range_from_metadata_len(
                split_len,
                bundle_metadata_len_range.start,
                bundle_metadata_len_bytes.as_slice(),
            )
        }

View on GitHub (pinned to a39730c5cd)

Solutions

1. Check the actual size of the split object in the object store and compare it with the expected size recorded at publish time; if smaller, the upload was truncated — delete the split and re-index the affected data.
2. Verify the path passed to read_split_footer/open_from_storage really points to a split file, not a hotcache or metadata file.
3. Re-upload or restore the split from a known-good copy and confirm the checksum matches before retrying.
4. If this reproduces on freshly indexed splits, inspect the uploader/packager stage for bugs writing incomplete bundles.

Example fix

// before: blindly open whatever the metastore points at
let footer = read_split_footer(storage, &split_path, len).await?;
// after: guard on size first
if len < SPLIT_FOOTER_TRAILER_NUM_BYTES as u64 {
    return Err(anyhow::anyhow!("split {} truncated ({} bytes), re-index required", split_path, len));
}
let footer = read_split_footer(storage, &split_path, len).await?;

let split_len = storage.file_num_bytes(&split_path).await?;
if split_len < quickwit_storage::SPLIT_FOOTER_TRAILER_NUM_BYTES as u64 {
    anyhow::bail!("split {} is truncated ({} bytes)", split_path, split_len);
}

fn split_size_plausible(split_len: u64) -> bool {
    split_len >= quickwit_storage::SPLIT_FOOTER_TRAILER_NUM_BYTES as u64
}

When it happens

Trigger: Calling read_split_footer or BundleStorage::open_from_storage on a split whose byte length (as reported by storage.file_num_bytes or the metastore) is less than SPLIT_FOOTER_TRAILER_NUM_BYTES. Typically the split upload was truncated, partially copied, or the wrong object was uploaded under the split path.

Common situations: Interrupted split uploads to S3/GCS leaving partial objects; manual copy/restore of split files that got truncated; a hotcache or staging file accidentally passed instead of the real split; metastore pointing at an object overwritten by a zero-byte or partial upload.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• failed to locate split footer after reading bundle metadata
• split tail exceeds split length
• split tail is longer than the split itself
• split footer starts after its trailer
• bundled file range starts after it ends

AI-assisted analysis of quickwit-oss/quickwit@a39730c5cd (2026-09-08). Data as JSON: /api/errors/92aaa77a44fd88c9. Report an issue: GitHub.