{"record":{"id":"70f883329ba81fe5","repo":"databendlabs/databend","slug":"hybrid-bitmap-small-set-size-overflow","errorCode":null,"errorMessage":"hybrid bitmap small set size overflow: {}","messagePattern":"hybrid bitmap small set size overflow: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"src/common/io/src/bitmap.rs","lineNumber":216,"sourceCode":"                rhs.iter().filter(|v| lhs.contains(**v)).count() as u64\n            }\n            (HybridBitmap::Small(lhs), HybridBitmap::Large(rhs)) => {\n                lhs.iter().filter(|v| rhs.contains(**v)).count() as u64\n            }\n            (HybridBitmap::Small(lhs), HybridBitmap::Small(rhs)) => {\n                small_intersection_len(lhs, rhs)\n            }\n        }\n    }\n\n    pub fn serialize_into<W: io::Write>(&self, mut writer: W) -> io::Result<()> {\n        writer.write_all(&HYBRID_MAGIC)?;\n        writer.write_all(&[HYBRID_VERSION])?;\n        match self {\n            HybridBitmap::Small(set) => {\n                writer.write_all(&[HYBRID_KIND_SMALL])?;\n                let len = u8::try_from(set.len()).map_err(|_| {\n                    io::Error::new(\n                        io::ErrorKind::InvalidData,\n                        format!(\"hybrid bitmap small set size overflow: {}\", set.len()),\n                    )\n                })?;\n                writer.write_all(&[len])?;\n                for value in set.iter() {\n                    writer.write_all(&value.to_le_bytes())?;\n                }\n                Ok(())\n            }\n            HybridBitmap::Large(tree) => {\n                writer.write_all(&[HYBRID_KIND_LARGE])?;\n                tree.serialize_into(writer)\n            }\n        }\n    }\n\n    pub fn iter(&self) -> HybridBitmapIter<'_> {","sourceCodeStart":198,"sourceCodeEnd":234,"githubUrl":"https://github.com/databendlabs/databend/blob/288d84d76e20a2f8f7173bda9691eb6ece301aa9/src/common/io/src/bitmap.rs#L198-L234","documentation":"HybridBitmap's Small variant stores at most u8::MAX (255) values. serialize_into writes the set length as one byte; if the small set holds more than 255 elements the u8::try_from conversion fails and serialization aborts with this InvalidData io error instead of silently truncating.","triggerScenarios":"Calling serialize_into on a HybridBitmap::Small whose set length is 256 or greater — an internal invariant violation, since larger sets should have been promoted to the large/roaring representation.","commonSituations":"A bug or manual construction that inserts more than 255 values into the Small variant, bypassing the size-based promotion logic.","solutions":["Check how the HybridBitmap was constructed; sets over 255 elements must use the large representation.","Ensure inserts go through the library's APIs that promote Small to Large when len exceeds 255.","If you hit this via aggregation/grouping code, report it — it indicates an internal invariant bug."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"if bitmap.len() > 255 {\n    return Err(anyhow!(\"small-set bitmap exceeds 255 elements\"));\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Only construct HybridBitmap::Small with at most 255 values.","Rely on library insert paths that promote Small to Large automatically.","Treat this error as an internal invariant bug and report it."],"tags":["serialization","bitmap","rust","overflow"],"backgroundTag":"value-out-of-range","analyzedSha":"288d84d76e20a2f8f7173bda9691eb6ece301aa9","analyzedAt":"2026-09-11T11:29:36.208Z","contentChangedAt":"2026-09-11T11:29:36.208Z","schemaVersion":2},"datasetVersion":"2026-09-16T04:17:20.429Z"}