{"record":{"id":"783a645f52e1a7e0","repo":"quickwit-oss/tantivy","slug":"err-to-string","errorCode":null,"errorMessage":"err.to_string()","messagePattern":"err\\.to_string\\(\\)","errorType":"exception","errorClass":"io::Error","httpStatus":null,"severity":"error","filePath":"src/store/compression_lz4_block.rs","lineNumber":17,"sourceCode":"use std::io::{self};\nuse std::mem;\n\nuse lz4_flex::{compress_into, decompress_into};\n\n#[inline]\n#[expect(clippy::uninit_vec)]\npub fn compress(uncompressed: &[u8], compressed: &mut Vec<u8>) -> io::Result<()> {\n    compressed.clear();\n    let maximum_output_size =\n        mem::size_of::<u32>() + lz4_flex::block::get_maximum_output_size(uncompressed.len());\n    compressed.reserve(maximum_output_size);\n    unsafe {\n        compressed.set_len(maximum_output_size);\n    }\n    let bytes_written = compress_into(uncompressed, &mut compressed[4..])\n        .map_err(|err| io::Error::new(io::ErrorKind::InvalidData, err.to_string()))?;\n    let num_bytes = uncompressed.len() as u32;\n    compressed[0..4].copy_from_slice(&num_bytes.to_le_bytes());\n    unsafe {\n        compressed.set_len(bytes_written + mem::size_of::<u32>());\n    }\n    Ok(())\n}\n\n#[inline]\n#[expect(clippy::uninit_vec)]\npub fn decompress(compressed: &[u8], decompressed: &mut Vec<u8>) -> io::Result<()> {\n    decompressed.clear();\n    let uncompressed_size_bytes: &[u8; 4] = compressed\n        .get(..4)\n        .ok_or(io::ErrorKind::InvalidData)?\n        .try_into()\n        .unwrap();\n    let uncompressed_size = u32::from_le_bytes(*uncompressed_size_bytes) as usize;","sourceCodeStart":1,"sourceCodeEnd":35,"githubUrl":"https://github.com/quickwit-oss/tantivy/blob/b5d8deb80c26924e6b007a5b1a7630f35ca64de4/src/store/compression_lz4_block.rs#L1-L35","documentation":"Thrown by compress() when lz4_flex's compress_into fails while compressing a doc store block. The underlying lz4 error is flattened into its string form and wrapped in an io::Error with ErrorKind::InvalidData. This library treats any lz4 compression failure as invalid input/state rather than retrying.","triggerScenarios":"Calling compress() (directly or via doc store writer) with a payload for which lz4_flex::block::compress_into reports an error, e.g. output buffer sizing issues or lz4 internal limits; the error surfaces when writing documents into a compressed doc store block.","commonSituations":"Writing extremely large or pathological byte payloads into the block compressor; mismatched tantivy/lz4_flex crate versions where maximum_output_size bookkeeping no longer matches the library's expectations; corrupted in-memory buffers passed in by the caller.","solutions":["Inspect the wrapped lz4 error message embedded in the io::Error to identify the exact lz4_flex failure","Verify the input byte slice being compressed is well-formed and of expected size","Check that lz4_flex and tantivy versions are compatible and up to date","If it reproduces on valid data, report it with a minimal reproducing payload"],"exampleFix":"// before\nlet bytes_written = compress_into(uncompressed, &mut compressed[4..])?;\n// after\nlet bytes_written = compress_into(uncompressed, &mut compressed[4..])\n    .map_err(|err| io::Error::new(io::ErrorKind::InvalidData, format!(\"lz4 compression failed: {err}\")))?;","handlingStrategy":"validation","validationCode":"fn ensure_compressible(data: &[u8]) -> io::Result<()> {\n    if data.is_empty() {\n        return Err(io::Error::new(io::ErrorKind::InvalidInput, \"cannot compress empty payload\"));\n    }\n    let max_out = std::mem::size_of::<u32>() + lz4_flex::block::get_maximum_output_size(data.len());\n    if max_out > isize::MAX as usize {\n        return Err(io::Error::new(io::ErrorKind::InvalidInput, \"payload too large for lz4 block compression\"));\n    }\n    Ok(())\n}","typeGuard":"fn is_compressible_input(data: &[u8]) -> bool {\n    !data.is_empty() && data.len() <= lz4_flex::block::MAX_INPUT_SIZE\n}","tryCatchPattern":"match compress(uncompressed, &mut compressed) {\n    Ok(()) => {},\n    Err(e) if e.kind() == io::ErrorKind::InvalidData => eprintln!(\"lz4 compression rejected input: {e}\"),\n    Err(e) => return Err(e),\n}","preventionTips":["Validate payload size against lz4 MAX_INPUT_SIZE before calling compress","Pin compatible lz4_flex and tantivy versions in your lockfile","Test compression of your real payload shapes (max document sizes) in CI","Handle the returned io::ErrorKind::InvalidData explicitly instead of unwrapping"],"tags":["io","compression","lz4","rust"],"backgroundTag":"lz4-compression-failed","analyzedSha":"b5d8deb80c26924e6b007a5b1a7630f35ca64de4","analyzedAt":"2026-09-05T13:20:51.521Z","contentChangedAt":"2026-09-05T13:20:51.521Z","schemaVersion":2},"datasetVersion":"2026-09-12T17:17:11.597Z"}