{"record":{"id":"7e861fcf4d78b897","repo":"databendlabs/databend","slug":"container-offset-exceeds-bitmap-data","errorCode":null,"errorMessage":"container offset exceeds bitmap data","messagePattern":"container offset exceeds bitmap data","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"src/common/io/src/bitmap/reader.rs","lineNumber":273,"sourceCode":"    }\n\n    pub fn bitmap_buf(&self) -> &[u8] {\n        &self.buf[4..]\n    }\n\n    pub(crate) fn container_offset(&self, i: usize) -> io::Result<usize> {\n        if i >= self.containers() {\n            return Err(Error::other(\"index out of range\"));\n        }\n        let offset_table_start = 12 + self.containers() * DESCRIPTION_BYTES;\n        let offset_pos = offset_table_start + i * OFFSET_BYTES;\n        if offset_pos + OFFSET_BYTES > self.buf.len() {\n            return Err(Error::other(\"offset table too short\"));\n        }\n        let mut reader = Cursor::new(&self.buf[offset_pos..]);\n        let offset = reader.read_u32::<LittleEndian>()? as usize;\n        if offset > self.bitmap_buf().len() {\n            return Err(Error::new(\n                ErrorKind::InvalidData,\n                \"container offset exceeds bitmap data\",\n            ));\n        }\n        Ok(offset)\n    }\n\n    pub fn container(&self, index: usize) -> io::Result<ContainerReader<'_>> {\n        let desc = self.description(index)?;\n        let offset = self.container_offset(index)?;\n        let cardinality = desc.cardinality();\n        let data = &self.bitmap_buf()[offset..];\n\n        // Validate container data length\n        let required_len = if cardinality <= ARRAY_LIMIT {\n            cardinality * 2\n        } else {\n            BITMAP_BYTES","sourceCodeStart":255,"sourceCodeEnd":291,"githubUrl":"https://github.com/databendlabs/databend/blob/288d84d76e20a2f8f7173bda9691eb6ece301aa9/src/common/io/src/bitmap/reader.rs#L255-L291","documentation":"BitmapReader::container_offset reads a container's 32-bit offset from the offset table and validates it against the bitmap data region. If the offset points beyond the end of bitmap_buf(), the encoded data is corrupt or inconsistent, so an InvalidData error is raised to avoid reading out of bounds.","triggerScenarios":"Decoding a bitmap whose stored container offset exceeds the bitmap data length — typically caused by truncated/corrupted payloads or a mismatched format version where the layout differs.","commonSituations":"Reading bitmaps produced by an incompatible writer version; manually edited or partially transferred serialized data.","solutions":["Verify the payload's magic/version matches the reader's expected format.","Re-transfer or re-serialize the bitmap; the stored bytes are inconsistent.","Check that the full buffer (header + descriptions + offsets + bitmap data) was passed to decode."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// verify payload header/version before decode\nif &bytes[0..4] != expected_magic {\n    return Err(anyhow!(\"bitmap format mismatch\"));\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Verify the bitmap magic/version before decoding foreign payloads.","Transfer serialized bitmaps atomically (checksums, complete writes).","Treat repeated occurrences as corrupt data: re-serialize from source."],"tags":["bitmap","deserialization","rust","corrupt-data"],"backgroundTag":"invalid-argument-value","analyzedSha":"288d84d76e20a2f8f7173bda9691eb6ece301aa9","analyzedAt":"2026-09-11T11:29:36.208Z","contentChangedAt":"2026-09-11T11:29:36.208Z","schemaVersion":2},"datasetVersion":"2026-09-16T04:17:20.429Z"}