quickwit-oss/tantivy · error · io::Error

InvalidData

InvalidData

Error message

Footer magic byte mismatch. File corrupted or index was created using old an tantivy version which is not supported anymore. Please use tantivy 0.15 or above to recreate the index.

What it means

After reading the footer's magic number, extract_footer compares it against FOOTER_MAGIC_NUMBER. A mismatch means the file was not written by a tantivy version that uses the current footer format (>= 0.15), or the file is simply not a tantivy index file.

Source

Thrown at src/directory/footer.rs:73

        if file.len() < 4 {
            return Err(io::Error::new(
                io::ErrorKind::UnexpectedEof,
                format!(
                    "File corrupted. The file is smaller than 4 bytes (len={}).",
                    file.len()
                ),
            ));
        }

        let footer_metadata_len = <(u32, u32)>::SIZE_IN_BYTES;
        let (footer_len, footer_magic_byte): (u32, u32) = file
            .slice_from_end(footer_metadata_len)
            .read_bytes()?
            .as_ref()
            .deserialize()?;

        if footer_magic_byte != FOOTER_MAGIC_NUMBER {
            return Err(io::Error::new(
                io::ErrorKind::InvalidData,
                "Footer magic byte mismatch. File corrupted or index was created using old an \
                 tantivy version which is not supported anymore. Please use tantivy 0.15 or above \
                 to recreate the index.",
            ));
        }

        if footer_len > FOOTER_MAX_LEN {
            return Err(io::Error::new(
                io::ErrorKind::InvalidData,
                format!(
                    "Footer seems invalid as it suggests a footer len of {footer_len}. File is \
                     corrupted, or the index was created with a different & old version of \
                     tantivy."
                ),
            ));
        }
        let total_footer_size = footer_len as usize + footer_metadata_len;

View on GitHub (pinned to b5d8deb80c)

Solutions

  1. Recreate the index with tantivy 0.15 or above (the error message's own remedy); old footer formats are not supported
  2. Restore an index that matches the tantivy version in use, or pin your tantivy version to one compatible with the existing index
  3. Verify the file is actually a tantivy segment (check the magic bytes with a hex dump)
  4. Re-index from the original documents if no compatible version/backup exists

Example fix

// before (Cargo.toml): opening a pre-0.15 index with a newer tantivy
tantivy = "0.21"
// after: reindex old data with 0.15+ first, or keep a matching version during migration
tantivy = "0.15" // migrate, then upgrade and re-save with the new version
Defensive patterns

Strategy: validation

Validate before calling

// check magic bytes before opening with an upgraded tantivy
let tail = /* read last 4 bytes of the file */;
let magic = u32::from_le_bytes(tail.try_into().unwrap());
if magic != 0x1af0e11c { /* wrong format: reindex or downgrade */ }

Try / catch

match extract_footer(file_slice) {
    Err(e) if e.kind() == io::ErrorKind::InvalidData
        && e.to_string().contains("magic byte mismatch") => {
        eprintln!("incompatible index version; reindex with tantivy 0.15+");
    }
    Err(e) => return Err(e),
    Ok(v) => v,
}

Prevention

When it happens

Trigger: Opening an index created with tantivy < 0.15; calling extract_footer on a non-tantivy file that happens to be in the index directory; byte-level corruption of the last bytes of a segment file.

Common situations: Upgrading tantivy across a major version and pointing the new version at an old index directory; mixing segment files from different tantivy versions; accidentally passing a log or temp file to extract_footer.

Related errors

AI-assisted analysis of quickwit-oss/tantivy@b5d8deb80c (2026-09-05). Data as JSON: /api/errors/eef6c7f531775d11. Report an issue: GitHub.