facebook/flow · error

failed to decompress

Error message

failed to decompress

What it means

Panics when lz4_flex::decompress_size_prepended rejects a byte slice. Every serialize_* function in this crate compresses with a size-prefixed LZ4 frame, so a failure means the stored bytes are not a valid frame: truncation, corruption, or data written by an incompatible producer. The input comes from flow's saved-state/heap files, which are caches of parsed artifacts.

Source

Thrown at rust_port/crates/flow_heap_serialization/src/lib.rs:31

use dupe::Dupe;
use flow_aloc::PackedALocTable;
use flow_common::docblock::Docblock;
use flow_imports_exports::exports::Exports;
use flow_imports_exports::imports::Imports;
use flow_parser::ast::Program;
use flow_parser::file_key::FileKey;
use flow_parser::loc::DESERIALIZE_FILE_KEY;
use flow_parser::loc::Loc;
use flow_parser_utils::file_sig::FileSig;
use flow_type_sig::packed_type_sig::Module;
use flow_type_sig::signature_error::TolerableError;

fn compress(data: &[u8]) -> Vec<u8> {
    lz4_flex::compress_prepend_size(data)
}

fn decompress(data: &[u8]) -> Vec<u8> {
    lz4_flex::decompress_size_prepended(data).expect("failed to decompress")
}

fn set_file_key(file_key: &FileKey) {
    DESERIALIZE_FILE_KEY.with(|k| *k.borrow_mut() = Some(file_key.dupe()));
}

fn clear_file_key() {
    DESERIALIZE_FILE_KEY.with(|k| *k.borrow_mut() = None);
}

pub fn serialize_ast(ast: &Program<Loc, Loc>) -> Vec<u8> {
    let bytes = bincode::serde::encode_to_vec(ast, bincode::config::legacy())
        .expect("failed to serialize AST");
    compress(&bytes)
}

pub fn deserialize_ast(file_key: &FileKey, bytes: &[u8]) -> Arc<Program<Loc, Loc>> {
    let decompressed = decompress(bytes);

View on GitHub (pinned to f88ac94bcf)

Solutions

1. Delete the affected saved-state/heap files so flow recomputes them from source (they are caches)
2. Make producers write atomically (write temp file, then rename) so partial files never appear under the final name
3. Verify the file's recorded size/length matches the actual file size before loading

Example fix

// before
let ast = flow_heap_serialization::deserialize_ast(&file_key, &bytes);

// after — treat decompression failure as a cache miss
let ast = match std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {
    flow_heap_serialization::deserialize_ast(&file_key, &bytes)
})) {
    Ok(ast) => ast,
    Err(_) => {
        std::fs::remove_file(&path).ok();
        reparse_and_save(&file_key, &path)?
    }
};

// Sanity-check the lz4 size prefix before decompressing a cached record
fn lz4_frame_plausible(data: &[u8]) -> bool {
    data.len() >= 4 && {
        let n = u32::from_le_bytes([data[0], data[1], data[2], data[3]]) as usize;
        n > 0 && n <= 512 * 1024 * 1024
    }
}

// Saved state is a cache: any decompress/decode panic means "miss"
match std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| deserialize(&bytes))) {
    Ok(v) => v,
    Err(_) => { std::fs::remove_file(&cache_path).ok(); recompute(&file_key)? }
}

When it happens

Trigger: Loading a heap/saved-state file that was truncated because the writer process was killed mid-write (writes are not atomic); a saved state produced by a different flow build whose compression layout changed; a file corrupted in transfer or by concurrent writers.

Common situations: flow killed (SIGKILL/OOM) while saving state, leaving a partial file that later loads fail on; copying saved states between hosts/versions with tools that mangle binary data; disk bit rot on long-lived caches.

Related errors

• invalid file index
• hh_load_heap: invalid magic number
• failed to deserialize AST
• failed to deserialize Docblock
• failed to deserialize PackedALocTable

AI-assisted analysis of facebook/flow@f88ac94bcf (2026-08-20). Data as JSON: /api/errors/8499ac54b2738307. Report an issue: GitHub.