{"record":{"id":"1314cdf357e47305","repo":"quickwit-oss/tantivy","slug":"sstable-corruption","errorCode":null,"errorMessage":"SSTable corruption","messagePattern":"SSTable corruption","errorType":"exception","errorClass":"io::Error","httpStatus":null,"severity":"error","filePath":"sstable/src/index/mod.rs","lineNumber":28,"sourceCode":"use crate::{TermOrdinal, common_prefix_len};\n\n#[derive(Debug, Clone)]\npub enum SSTableIndex {\n    V2(v2::SSTableIndex),\n    V3(v3::SSTableIndexV3),\n    V3Empty(v3::SSTableIndexV3Empty),\n}\n\nimpl SSTableIndex {\n    pub(crate) fn open(\n        version: u32,\n        index_offset: u64,\n        index_bytes: OwnedBytes,\n    ) -> io::Result<Self> {\n        let index = match version {\n            2 => {\n                SSTableIndex::V2(v2::SSTableIndex::load(index_bytes).map_err(|_| {\n                    io::Error::new(io::ErrorKind::InvalidData, \"SSTable corruption\")\n                })?)\n            }\n            3 => {\n                let (index_bytes, mut footerv3_len_bytes) = index_bytes.rsplit(8);\n                let store_offset = u64::deserialize(&mut footerv3_len_bytes)?;\n                if store_offset != 0 {\n                    SSTableIndex::V3(v3::SSTableIndexV3::load(index_bytes, store_offset).map_err(\n                        |_| io::Error::new(io::ErrorKind::InvalidData, \"SSTable corruption\"),\n                    )?)\n                } else {\n                    // if store_offset is zero, there is no index, so we build a pseudo-index\n                    // assuming a single block of sstable covering everything.\n                    SSTableIndex::V3Empty(v3::SSTableIndexV3Empty::load(index_offset as usize))\n                }\n            }\n            _ => {\n                return Err(io::Error::other(format!(\n                    \"Unsupported sstable version, expected one of [2, 3], found {version}\"","sourceCodeStart":10,"sourceCodeEnd":46,"githubUrl":"https://github.com/quickwit-oss/tantivy/blob/b5d8deb80c26924e6b007a5b1a7630f35ca64de4/sstable/src/index/mod.rs#L10-L46","documentation":"When opening an SSTable with format version 2, the serialized index footer/index bytes fail to parse (v2::SSTableIndex::load returns Err), so open() converts that into io::Error of kind InvalidData with the message \"SSTable corruption\". It means the index portion of the file is not valid for the declared version — usually the file is truncated, partially written, or not actually the version it claims.","triggerScenarios":"Calling SSTable::open on a version-2 sstable whose index_bytes cannot be deserialized by v2::SSTableIndex::load: truncated file, wrong offset passed for index_bytes, or bytes from a different/newer format labeled as v2.","commonSituations":"Interrupted index writes (crash/power loss) leaving a partial segment; copying index files without the footer bytes; opening files produced by an incompatible tantivy version; corrupted downloads or bad mmap ranges.","solutions":["Verify the segment file is complete and untruncated; restore from backup or re-index the affected segment.","Re-check how index_offset/index_bytes are derived — a wrong offset yields misaligned bytes that fail to load.","Ensure the reading library version matches the version that wrote the index; upgrade tantivy if the index was written by a newer release.","Run a corruption check / segment merge to rebuild the index from the stored docs if supported."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"// Before opening: check file size covers the expected index region\nfn index_bytes_intact(file_len: u64, index_offset: u64, min_index_len: u64) -> bool {\n    file_len >= index_offset.saturating_add(min_index_len)\n}","typeGuard":null,"tryCatchPattern":"// catch InvalidData \"SSTable corruption\" and fall back to re-indexing\nmatch SSTable::open(...) {\n    Err(e) if e.kind() == io::ErrorKind::InvalidData\n        && e.to_string() == \"SSTable corruption\" => {\n        // restore from backup or rebuild segment\n    }\n    other => other?,\n}","preventionTips":["Use checksummed/copy-safe transfers for segment files","Never kill the writer process mid-commit; rely on proper commit/flush","Pin tantivy versions between writer and reader","Keep backups of segments before manual manipulation"],"tags":["rust","sstable","corruption","io"],"backgroundTag":"sstable-index-corruption","analyzedSha":"b5d8deb80c26924e6b007a5b1a7630f35ca64de4","analyzedAt":"2026-09-05T13:20:51.521Z","contentChangedAt":"2026-09-05T13:20:51.521Z","schemaVersion":2},"datasetVersion":"2026-09-12T17:17:11.597Z"}