quickwit-oss/tantivy · critical

unknown compressor id {id:?}

Error message

unknown compressor id {id:?}

What it means

Decompressor::from_id maps the compressor id byte stored in a segment's doc-store footer to a decompression backend (0=None, 1=Lz4, 4=Zstd). Lz4 and Zstd branches only exist when their cargo feature is compiled in, so an id recorded on disk whose codec is unavailable (or a garbage id) hits the panic. It means the segment cannot be decompressed by this build.

Source

Thrown at src/store/decompressors.rs:40

        match compressor {
            Compressor::None => Decompressor::None,
            #[cfg(feature = "lz4-compression")]
            Compressor::Lz4 => Decompressor::Lz4,
            #[cfg(feature = "zstd-compression")]
            Compressor::Zstd(_) => Decompressor::Zstd,
        }
    }
}

impl Decompressor {
    pub(crate) fn from_id(id: u8) -> Decompressor {
        match id {
            0 => Decompressor::None,
            #[cfg(feature = "lz4-compression")]
            1 => Decompressor::Lz4,
            #[cfg(feature = "zstd-compression")]
            4 => Decompressor::Zstd,
            _ => panic!("unknown compressor id {id:?}"),
        }
    }

    pub(crate) fn get_id(&self) -> u8 {
        match self {
            Self::None => 0,
            #[cfg(feature = "lz4-compression")]
            Self::Lz4 => 1,
            #[cfg(feature = "zstd-compression")]
            Self::Zstd => 4,
        }
    }

    pub(crate) fn decompress(&self, compressed_block: &[u8]) -> io::Result<Vec<u8>> {
        let mut decompressed_block = vec![];
        self.decompress_into(compressed_block, &mut decompressed_block)?;
        Ok(decompressed_block)
    }

View on GitHub (pinned to b5d8deb80c)

Solutions

1. Enable the matching codec feature in Cargo.toml (features = ["lz4-compression"] or ["zstd-compression"]) and rebuild
2. Align writer and reader builds to the same compression feature set
3. Re-index or rewrite segments using an available compressor (e.g. None) if the build cannot change
4. Verify segment file integrity if the id byte itself is corrupt

Example fix

// before (Cargo.toml)
tantivy = "0.22"
// after
tantivy = { version = "0.22", features = ["lz4-compression", "zstd-compression"] }

fn codec_available(id: u8) -> bool {
    match id {
        0 => true,
        1 => cfg!(feature = "lz4-compression"),
        4 => cfg!(feature = "zstd-compression"),
        _ => false,
    }
}
// check footer compressor id before opening: assert!(codec_available(id));

When it happens

Trigger: Opening a doc store whose footer records compressor id 1 while lz4-compression is disabled, or id 4 while zstd-compression is disabled; reading a segment written by a build with a codec feature this build lacks; a corrupted/newer file with an unknown id byte.

Common situations: Writer and reader compiled with different Cargo feature sets; --no-default-features builds silently dropping codec support; index files copied between services with different dependency configs; truncated/hand-edited segment files.

Related errors

• actual doc store version: {doc_store_version}, max_supported
• zstd-compression feature is not enabled
• FastFieldsPlugin is a built-in; use FastFieldsPluginWriter::
• err.to_string()
• doc store block not completely decompressed, data corruption

AI-assisted analysis of quickwit-oss/tantivy@b5d8deb80c (2026-09-05). Data as JSON: /api/errors/17f3c71cfd64f121. Report an issue: GitHub.