{"record":{"id":"6e358493e528f3d6","repo":"databendlabs/databend","slug":"fst-data-is-corrupted","errorCode":null,"errorMessage":"Fst data is corrupted: {:?}","messagePattern":"Fst data is corrupted: (.+?)","errorType":"exception","errorClass":"InvalidData","httpStatus":null,"severity":"error","filePath":"src/query/storages/fuse/src/io/read/inverted_index/inverted_index_reader.rs","lineNumber":277,"sourceCode":"            if let Some(term_col_meta) = inverted_index_meta_map.remove(&term_col_name) {\n                let term_range = term_col_meta.offset..(term_col_meta.offset + term_col_meta.len);\n                columns.push((term_col_name, term_range));\n            }\n        }\n\n        let column_files =\n            legacy_load_inverted_index_files(settings, columns, index_path, &self.dal).await?;\n        let mut column_files_map = column_files\n            .into_iter()\n            .map(|f| (f.name.clone(), f.data.clone()))\n            .collect::<HashMap<_, _>>();\n\n        let mut fst_maps = HashMap::with_capacity(field_ids.len());\n        for field_id in field_ids {\n            let fst_col_name = format!(\"fst-{}\", field_id);\n            let fst = if let Some(fst_data) = column_files_map.remove(&fst_col_name) {\n                Fst::new(fst_data).map_err(|err| {\n                    std::io::Error::new(\n                        std::io::ErrorKind::InvalidData,\n                        format!(\"Fst data is corrupted: {:?}\", err),\n                    )\n                })?\n            } else {\n                // If the FST data does not exist, create an empty FST.\n                // This means that the field does not have any valid terms.\n                let builder = tantivy_fst::MapBuilder::memory();\n                let bytes = builder.into_inner().unwrap();\n                let fst_data = OwnedBytes::new(bytes);\n                Fst::new(fst_data).unwrap()\n            };\n            let fst_map = tantivy_fst::Map::from(fst);\n            fst_maps.insert(*field_id, fst_map);\n        }\n\n        // 2. check whether query is matched in the fsts.\n        let mut matched_terms = HashMap::new();","sourceCodeStart":259,"sourceCodeEnd":295,"githubUrl":"https://github.com/databendlabs/databend/blob/288d84d76e20a2f8f7173bda9691eb6ece301aa9/src/query/storages/fuse/src/io/read/inverted_index/inverted_index_reader.rs#L259-L295","documentation":"custom_search_impl builds Fst maps from stored column data; if Fst::new fails to parse the stored FST bytes it wraps the error in an io::Error(InvalidData) reading 'Fst data is corrupted: {:?}'. This means the persisted FST (finite-state transducer) for an inverted-index field is unreadable — truncated, empty-but-marked-present, or written by an incompatible fst version. The search via search() cannot proceed for that field.","triggerScenarios":"Inverted-index search where the column file named 'fst-{field_id}' exists but its bytes fail Fst::new validation (bad magic/version/checksum, truncation, empty vector).","commonSituations":"Partially written index segments after a crash; corrupted snapshot files; index format version skew between writer and reader; manual file truncation during copy.","solutions":["Verify the fst-{field_id} column blob is complete (compare against expected size / re-read from storage)","Rebuild the inverted index for the affected fields so the FST is regenerated","Check that reader and writer use compatible fst crate versions / index format versions","Restore the segment/index files from snapshot or backup"],"exampleFix":"// before: assuming present blob is valid\nlet fst = Fst::new(fst_data).map_err(|err| io::Error::new(ErrorKind::InvalidData, format!(\"Fst data is corrupted: {:?}\", err)))?;\n// after: treat empty/corrupt as absent and fall back\nlet fst = Fst::new(fst_data).unwrap_or_else(|_| Fst::from_iter_str(vec![String::new()])); // empty FST fallback","handlingStrategy":"try-catch","validationCode":"// Fst::new validates magic/version; pre-check presence and non-emptiness\nif fst_blob.is_empty() { return Err(\"fst column is empty; index needs rebuild\"); }","typeGuard":"fn fst_data_ok(b: &[u8]) -> bool { !b.is_empty() && b.len() >= 8 }","tryCatchPattern":"let fst = match Fst::new(fst_data) {\n    Ok(f) => f,\n    Err(err) => {\n        log::warn!(\"fst-{} corrupted: {:?}; rebuilding index\", field_id, err);\n        rebuild_inverted_index(field_id).await?\n    }\n};","preventionTips":["Snapshot files only after index writes are fsynced and complete","Record FST sizes/checksums in the index meta and verify before Fst::new","Keep fst crate versions consistent between index writers and readers","Schedule periodic index verification/rebuild jobs"],"tags":["inverted-index","fst","corruption","rust","io"],"backgroundTag":"checksum-mismatch","analyzedSha":"288d84d76e20a2f8f7173bda9691eb6ece301aa9","analyzedAt":"2026-09-11T11:29:36.208Z","contentChangedAt":"2026-09-11T11:29:36.208Z","schemaVersion":2},"datasetVersion":"2026-09-16T04:17:20.429Z"}