quickwit-oss/quickwit · error

hot directory metadata's magic number does not match

Error message

hot directory metadata's magic number does not match

What it means

Versioned components (e.g. hot directory metadata) persist an 8-byte header: a 4-byte LE magic number followed by a version code. During deserialization, if the first 4 bytes don't equal the component's MAGIC_NUMBER, the bytes are not a file this reader understands and it bails before even checking the version. Guards against reading unrelated or corrupted files.

Source

Thrown at quickwit/quickwit-storage/src/versioned_component.rs:100

///
/// (This function is not part of the trait to make it private.)
fn try_read_version<V: VersionedComponent>(bytes: &mut OwnedBytes) -> anyhow::Result<V> {
    let mut header_bytes: [u8; 8] = [0u8; 8];
    bytes
        .read_exact(&mut header_bytes[..])
        .with_context(|| format!("failed to read header for {}", V::component_name()))?;
    try_deserialize_from_bytes::<V>(header_bytes)
}

/// Deserialize the header from 8 bytes.
/// An error is returned if the magic number does not match,
/// or if the version is unsupported.
///
/// (This function is not part of the trait to make it private.)
fn try_deserialize_from_bytes<V: VersionedComponent>(header_bytes: [u8; 8]) -> anyhow::Result<V> {
    let magic_number = u32::from_le_bytes(header_bytes[0..4].try_into().unwrap());
    if magic_number != V::MAGIC_NUMBER {
        anyhow::bail!("hot directory metadata's magic number does not match");
    }
    let version_code: u32 = u32::from_le_bytes(header_bytes[4..8].try_into().unwrap());
    V::try_from_version_code_impl(version_code).with_context(|| {
        format!(
            "version code {} is not supported for {}",
            version_code,
            V::component_name()
        )
    })
}

#[cfg(test)]
mod tests {
    use tantivy::directory::OwnedBytes;

    use crate::VersionedComponent;

    #[derive(Copy, Clone, Default)]

View on GitHub (pinned to a39730c5cd)

Solutions

1. Delete the corrupted hotcache/metadata file and let quickwit rebuild it from the split.
2. Verify the file was produced by the same quickwit version writing the same component format.
3. Check storage/disk health for corruption if this recurs.

let mut header = [0u8; 8];
file.read_exact(&mut header)?;
let magic = u32::from_le_bytes(header[0..4].try_into()?);
if magic != EXPECTED_MAGIC { /* rebuild hotcache instead of loading */ }

match try_deserialize(bytes) {
    Err(e) if e.to_string().contains("magic number") => rebuild_metadata(),
    other => other?,
}

When it happens

Trigger: Opening a hotcache/hot-directory metadata file whose header was corrupted, was written by a different component, or is a truncated/zeroed file so the magic number check fails.

Common situations: Disk corruption or partial writes on the hotcache directory; pointing a reader at the wrong file; manually copied/moved files between incompatible quickwit versions or storage layouts.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

Related errors

• invalid boolean value
• source `{}` is defined more than once
• failed to locate split footer after reading bundle metadata
• missing recovery index UID
• missing recovery doc mapping UID

AI-assisted analysis of quickwit-oss/quickwit@a39730c5cd (2026-09-08). Data as JSON: /api/errors/b930861a288e160e. Report an issue: GitHub.