{"record":{"id":"8499ac54b2738307","repo":"facebook/flow","slug":"failed-to-decompress","errorCode":null,"errorMessage":"failed to decompress","messagePattern":"failed to decompress","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"rust_port/crates/flow_heap_serialization/src/lib.rs","lineNumber":31,"sourceCode":"use dupe::Dupe;\nuse flow_aloc::PackedALocTable;\nuse flow_common::docblock::Docblock;\nuse flow_imports_exports::exports::Exports;\nuse flow_imports_exports::imports::Imports;\nuse flow_parser::ast::Program;\nuse flow_parser::file_key::FileKey;\nuse flow_parser::loc::DESERIALIZE_FILE_KEY;\nuse flow_parser::loc::Loc;\nuse flow_parser_utils::file_sig::FileSig;\nuse flow_type_sig::packed_type_sig::Module;\nuse flow_type_sig::signature_error::TolerableError;\n\nfn compress(data: &[u8]) -> Vec<u8> {\n    lz4_flex::compress_prepend_size(data)\n}\n\nfn decompress(data: &[u8]) -> Vec<u8> {\n    lz4_flex::decompress_size_prepended(data).expect(\"failed to decompress\")\n}\n\nfn set_file_key(file_key: &FileKey) {\n    DESERIALIZE_FILE_KEY.with(|k| *k.borrow_mut() = Some(file_key.dupe()));\n}\n\nfn clear_file_key() {\n    DESERIALIZE_FILE_KEY.with(|k| *k.borrow_mut() = None);\n}\n\npub fn serialize_ast(ast: &Program<Loc, Loc>) -> Vec<u8> {\n    let bytes = bincode::serde::encode_to_vec(ast, bincode::config::legacy())\n        .expect(\"failed to serialize AST\");\n    compress(&bytes)\n}\n\npub fn deserialize_ast(file_key: &FileKey, bytes: &[u8]) -> Arc<Program<Loc, Loc>> {\n    let decompressed = decompress(bytes);","sourceCodeStart":13,"sourceCodeEnd":49,"githubUrl":"https://github.com/facebook/flow/blob/f88ac94bcf6992f5d5a158854d94613ebb92c6e6/rust_port/crates/flow_heap_serialization/src/lib.rs#L13-L49","documentation":"Panics when lz4_flex::decompress_size_prepended rejects a byte slice. Every serialize_* function in this crate compresses with a size-prefixed LZ4 frame, so a failure means the stored bytes are not a valid frame: truncation, corruption, or data written by an incompatible producer. The input comes from flow's saved-state/heap files, which are caches of parsed artifacts.","triggerScenarios":"Loading a heap/saved-state file that was truncated because the writer process was killed mid-write (writes are not atomic); a saved state produced by a different flow build whose compression layout changed; a file corrupted in transfer or by concurrent writers.","commonSituations":"flow killed (SIGKILL/OOM) while saving state, leaving a partial file that later loads fail on; copying saved states between hosts/versions with tools that mangle binary data; disk bit rot on long-lived caches.","solutions":["Delete the affected saved-state/heap files so flow recomputes them from source (they are caches)","Make producers write atomically (write temp file, then rename) so partial files never appear under the final name","Verify the file's recorded size/length matches the actual file size before loading"],"exampleFix":"// before\nlet ast = flow_heap_serialization::deserialize_ast(&file_key, &bytes);\n\n// after — treat decompression failure as a cache miss\nlet ast = match std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {\n    flow_heap_serialization::deserialize_ast(&file_key, &bytes)\n})) {\n    Ok(ast) => ast,\n    Err(_) => {\n        std::fs::remove_file(&path).ok();\n        reparse_and_save(&file_key, &path)?\n    }\n};","handlingStrategy":"fallback","validationCode":"// Sanity-check the lz4 size prefix before decompressing a cached record\nfn lz4_frame_plausible(data: &[u8]) -> bool {\n    data.len() >= 4 && {\n        let n = u32::from_le_bytes([data[0], data[1], data[2], data[3]]) as usize;\n        n > 0 && n <= 512 * 1024 * 1024\n    }\n}","typeGuard":null,"tryCatchPattern":"// Saved state is a cache: any decompress/decode panic means \"miss\"\nmatch std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| deserialize(&bytes))) {\n    Ok(v) => v,\n    Err(_) => { std::fs::remove_file(&cache_path).ok(); recompute(&file_key)? }\n}","preventionTips":["Write heap/saved-state files atomically: temp file then rename","Store an outer length/checksum and verify before loading","Namespace cache directories by flow version so cross-version reads cannot happen"],"tags":["rust","lz4","compression","corruption","saved-state"],"backgroundTag":"lz4-decompression-failed","analyzedSha":"f88ac94bcf6992f5d5a158854d94613ebb92c6e6","analyzedAt":"2026-08-20T10:41:37.992Z","schemaVersion":2},"datasetVersion":"2026-08-23T11:17:13.642Z"}