{"record":{"id":"ec7702c42fecf825","repo":"quickwit-oss/tantivy","slug":"invaliddata-ec7702","errorCode":"InvalidData","errorMessage":"Reach end of buffer while reading VInt","messagePattern":"Reach end of buffer while reading VInt","errorType":"error_code","errorClass":"io::Error","httpStatus":null,"severity":"error","filePath":"common/src/vint.rs","lineNumber":47,"sourceCode":"    }\n\n    #[allow(clippy::unbuffered_bytes)]\n    fn deserialize<R: Read>(reader: &mut R) -> io::Result<Self> {\n        #[allow(clippy::unbuffered_bytes)]\n        let mut bytes = reader.bytes();\n        let mut result = 0u128;\n        let mut shift = 0u64;\n        loop {\n            match bytes.next() {\n                Some(Ok(b)) => {\n                    result |= u128::from(b % 128u8) << shift;\n                    if b >= STOP_BIT {\n                        return Ok(VIntU128(result));\n                    }\n                    shift += 7;\n                }\n                _ => {\n                    return Err(io::Error::new(\n                        io::ErrorKind::InvalidData,\n                        \"Reach end of buffer while reading VInt\",\n                    ));\n                }\n            }\n        }\n    }\n}\n\n///   Wrapper over a `u64` that serializes as a variable int.\n#[derive(Clone, Copy, Debug, Eq, PartialEq)]\npub struct VInt(pub u64);\n\nconst STOP_BIT: u8 = 128;\n\n#[inline]\npub fn serialize_vint_u32(val: u32, buf: &mut [u8; 8]) -> &[u8] {\n    const START_2: u64 = 1 << 7;","sourceCodeStart":29,"sourceCodeEnd":65,"githubUrl":"https://github.com/quickwit-oss/tantivy/blob/b5d8deb80c26924e6b007a5b1a7630f35ca64de4/common/src/vint.rs#L29-L65","documentation":"VInt/U128 varint deserialization reads bytes until one has the STOP_BIT set. If the buffer ends before a stop byte is found, deserialize raises InvalidData 'Reach end of buffer while reading VInt' — the varint is truncated. This indicates truncation or a misaligned read position.","triggerScenarios":"deserialize (read_u64/unsafe_read_u64-style varint readers in vint.rs) consuming a byte slice that ends while the high bit of the current byte is still clear — truncated buffer, wrong read offset, or corrupt length prefixes.","commonSituations":"Truncated segment/footer files from incomplete writes or downloads; reading a varint at an offset one byte off (misalignment makes the terminator fall outside the slice); corrupt length-prefixed blocks.","solutions":["Validate the byte slice length against the expected varint/record boundaries before deserializing","Verify file integrity (checksums/footers) to catch truncation","Check that read offsets are correctly aligned with preceding serialized fields","Re-obtain or re-index the corrupted data"],"exampleFix":"// before\nlet v = VIntU128::deserialize(&mut bytes)?;\n// after\nif bytes.as_slice().is_empty() { return Err(io::Error::new(io::ErrorKind::UnexpectedEof, \"no bytes for VInt\")); }\nlet v = VIntU128::deserialize(&mut bytes)?;","handlingStrategy":"validation","validationCode":"fn has_room_for_varint(bytes: &[u8]) -> bool {\n    // ensure at least one remaining byte; full validation is walking until STOP_BIT\n    !bytes.is_empty()\n}\n// better: walk: for (i, b) in bytes.iter().enumerate() { if b >= 128 { /* ok */ } }","typeGuard":null,"tryCatchPattern":"match VIntU128::deserialize(&mut bytes.clone()) {\n    Err(e) if e.kind() == io::ErrorKind::InvalidData && e.to_string().contains(\"Reach end of buffer while reading VInt\") => {\n        // truncated buffer: verify file length/checksum or fix read offset\n    }\n    other => other?,\n}","preventionTips":["Validate buffers contain a STOP_BIT-terminated varint before reading","Verify file checksums/footers to catch truncation early","Recheck offset arithmetic for preceding variable-length fields","Persist and compare expected lengths at write time to detect partial writes"],"tags":["io","deserialization","varint","truncation"],"backgroundTag":"truncated-varint","analyzedSha":"b5d8deb80c26924e6b007a5b1a7630f35ca64de4","analyzedAt":"2026-09-05T13:20:51.521Z","contentChangedAt":"2026-09-05T13:20:51.521Z","schemaVersion":2},"datasetVersion":"2026-09-12T17:17:11.597Z"}