databendlabs/databend · error

container offset exceeds bitmap data

Error message

container offset exceeds bitmap data

What it means

BitmapReader::container_offset reads a container's 32-bit offset from the offset table and validates it against the bitmap data region. If the offset points beyond the end of bitmap_buf(), the encoded data is corrupt or inconsistent, so an InvalidData error is raised to avoid reading out of bounds.

Solutions

1. Verify the payload's magic/version matches the reader's expected format.
2. Re-transfer or re-serialize the bitmap; the stored bytes are inconsistent.
3. Check that the full buffer (header + descriptions + offsets + bitmap data) was passed to decode.

// verify payload header/version before decode
if &bytes[0..4] != expected_magic {
    return Err(anyhow!("bitmap format mismatch"));
}

When it happens

Trigger: Decoding a bitmap whose stored container offset exceeds the bitmap data length — typically caused by truncated/corrupted payloads or a mismatched format version where the layout differs.

Common situations: Reading bitmaps produced by an incompatible writer version; manually edited or partially transferred serialized data.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• data is truncated or invalid
• Unimplemented deserialize DataSourceMeta
• expected a non-NaN float
• hybrid bitmap small set size overflow
• index out of range

AI-assisted analysis of databendlabs/databend@288d84d76e (2026-09-11). Data as JSON: /api/errors/7e861fcf4d78b897. Report an issue: GitHub.

Appendix: source

Thrown at src/common/io/src/bitmap/reader.rs:273

    }

    pub fn bitmap_buf(&self) -> &[u8] {
        &self.buf[4..]
    }

    pub(crate) fn container_offset(&self, i: usize) -> io::Result<usize> {
        if i >= self.containers() {
            return Err(Error::other("index out of range"));
        }
        let offset_table_start = 12 + self.containers() * DESCRIPTION_BYTES;
        let offset_pos = offset_table_start + i * OFFSET_BYTES;
        if offset_pos + OFFSET_BYTES > self.buf.len() {
            return Err(Error::other("offset table too short"));
        }
        let mut reader = Cursor::new(&self.buf[offset_pos..]);
        let offset = reader.read_u32::<LittleEndian>()? as usize;
        if offset > self.bitmap_buf().len() {
            return Err(Error::new(
                ErrorKind::InvalidData,
                "container offset exceeds bitmap data",
            ));
        }
        Ok(offset)
    }

    pub fn container(&self, index: usize) -> io::Result<ContainerReader<'_>> {
        let desc = self.description(index)?;
        let offset = self.container_offset(index)?;
        let cardinality = desc.cardinality();
        let data = &self.bitmap_buf()[offset..];

        // Validate container data length
        let required_len = if cardinality <= ARRAY_LIMIT {
            cardinality * 2
        } else {
            BITMAP_BYTES

View on GitHub (pinned to 288d84d76e)