{"record":{"id":"fa6d15c353f3494b","repo":"databendlabs/databend","slug":"data-is-truncated-or-invalid","errorCode":null,"errorMessage":"data is truncated or invalid","messagePattern":"data is truncated or invalid","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"src/common/io/src/bitmap/reader.rs","lineNumber":222,"sourceCode":"        }\n\n        let last_container = (containers - 1) as i64;\n        reader.seek_relative(last_container * DESCRIPTION_BYTES as i64 + 2)?;\n        let last_cardinality = reader.read_u16::<LittleEndian>()? as usize + 1;\n\n        reader.seek_relative(last_container * OFFSET_BYTES as i64)?;\n        let last_offset = reader.read_u32::<LittleEndian>()?;\n\n        let size = 4\n            + last_offset as usize\n            + if last_cardinality <= ARRAY_LIMIT {\n                2 * last_cardinality\n            } else {\n                BITMAP_BYTES\n            };\n\n        if buf.len() < size {\n            Err(Error::new(\n                ErrorKind::UnexpectedEof,\n                \"data is truncated or invalid\",\n            ))\n        } else {\n            Ok(BitmapReader {\n                prefix,\n                containers,\n                buf: &buf[..size],\n            })\n        }\n    }\n\n    pub fn containers(&self) -> usize {\n        self.containers as usize\n    }\n\n    pub fn prefix(&self) -> u32 {\n        self.prefix","sourceCodeStart":204,"sourceCodeEnd":240,"githubUrl":"https://github.com/databendlabs/databend/blob/288d84d76e20a2f8f7173bda9691eb6ece301aa9/src/common/io/src/bitmap/reader.rs#L204-L240","documentation":"BitmapReader::decode validates that the input buffer is at least as long as the container area requires (computed from the last container's cardinality, or BITMAP_BYTES for array containers). If buf is shorter, the encoded bitmap data is incomplete or corrupt and decoding fails with UnexpectedEof.","triggerScenarios":"Calling BitmapReader::decode (e.g. via Bitmap deserialization) with a byte slice that was truncated in transit, sliced incorrectly, or produced by an incompatible encoder.","commonSituations":"Network/storage truncation of serialized bitmaps; passing a header-only slice instead of the full payload; reading a file written by a different bitmap format version.","solutions":["Verify the full serialized bitmap bytes were written/read (check write completeness and file sizes).","Confirm the byte source matches the expected bitmap encoding/version (check magic/version bytes upstream).","Re-export or recompute the bitmap data if the payload is confirmed corrupt."],"exampleFix":"// before\nlet reader = BitmapReader::decode(&partial_bytes)?;\n\n// after\nif partial_bytes.len() < expected_min_size {\n    return Err(anyhow!(\"bitmap payload truncated: {} < {}\", partial_bytes.len(), expected_min_size));\n}\nlet reader = BitmapReader::decode(&partial_bytes)?;","handlingStrategy":"validation","validationCode":"if bytes.len() < expected_min_bitmap_size {\n    return Err(anyhow!(\"bitmap payload too short\"));\n}","typeGuard":null,"tryCatchPattern":"match BitmapReader::decode(&bytes) {\n    Ok(r) => r,\n    Err(e) if e.kind() == io::ErrorKind::UnexpectedEof => {\n        log::warn!(\"truncated bitmap payload\"); return Err(e.into());\n    }\n    Err(e) => return Err(e.into()),\n}","preventionTips":["Check write completeness (flush/close files, verify network read lengths).","Store payload lengths alongside serialized bitmaps and verify before decode.","Keep producer and consumer bitmap format versions in sync."],"tags":["bitmap","deserialization","rust","truncated-data"],"backgroundTag":"checksum-mismatch","analyzedSha":"288d84d76e20a2f8f7173bda9691eb6ece301aa9","analyzedAt":"2026-09-11T11:29:36.208Z","contentChangedAt":"2026-09-11T11:29:36.208Z","schemaVersion":2},"datasetVersion":"2026-09-16T04:17:20.429Z"}