{"record":{"id":"7d9e7ae0a9a0b26e","repo":"risingwavelabs/risingwave","slug":"prost-payload-length-exceeds-remaining-buffer","errorCode":null,"errorMessage":"prost payload length {} exceeds remaining buffer {}","messagePattern":"prost payload length (.+?) exceeds remaining buffer (.+?)","errorType":"exception","errorClass":"BackupError","httpStatus":null,"severity":"error","filePath":"src/storage/backup/src/meta_snapshot_v1.rs","lineNumber":238,"sourceCode":"            cluster_id,\n            subscription,\n            secret,\n        })\n    }\n\n    fn encode_prost_message(message: &impl prost::Message, buf: &mut impl BufMut) {\n        let encoded_message = message.encode_to_vec();\n        buf.put_u32_le(encoded_message.len() as u32);\n        buf.put_slice(&encoded_message);\n    }\n\n    fn decode_prost_message<T>(buf: &mut &[u8]) -> BackupResult<T>\n    where\n        T: prost::Message + Default,\n    {\n        let len = read_u32_le(buf)? as usize;\n        if buf.remaining() < len {\n            return Err(BackupError::Other(anyhow::anyhow!(\n                \"prost payload length {} exceeds remaining buffer {}\",\n                len,\n                buf.remaining()\n            )));\n        }\n        let v = buf[..len].to_vec();\n        buf.advance(len);\n        T::decode(v.as_slice()).map_err(|e| BackupError::Decoding(e.into()))\n    }\n\n    fn encode_prost_message_list(messages: &[&impl prost::Message], buf: &mut impl BufMut) {\n        buf.put_u32_le(messages.len() as u32);\n        for message in messages {\n            Self::encode_prost_message(*message, buf);\n        }\n    }\n\n    fn decode_prost_message_list<T>(buf: &mut &[u8]) -> BackupResult<Vec<T>>","sourceCodeStart":220,"sourceCodeEnd":256,"githubUrl":"https://github.com/risingwavelabs/risingwave/blob/6469eb736d691e8e9b8a419a57edd6429ca77417/src/storage/backup/src/meta_snapshot_v1.rs#L220-L256","documentation":"decode_prost_message reads a u32 length prefix then requires that many bytes in the buffer; if buf.remaining() < len it throws \"prost payload length {} exceeds remaining buffer {}\". The declared protobuf message size exceeds the data actually available, so the snapshot is truncated or its length field is corrupt/foreign.","triggerScenarios":"Decoding any prost-encoded metadata list inside the snapshot (tables, hummock sequences, system parameters) when the section bytes end before the declared length — corrupted buffer, truncated object, or misparsed framing causing a bogus length.","commonSituations":"A backup object truncated mid-section; a snapshot body parsed with a wrong offset so random bytes become a huge length prefix; endianness/framing mismatch from hand-crafted test data; older-format sections decoded by a newer reader.","solutions":["Restore from an intact backup — compare stored object size and checksum with the manifest.","Verify the parse offset: ensure read_snapshot_header and prior section skips consumed the correct byte counts before this decode.","Check the snapshot format_version matches the decoder version in use.","Rebuild test payloads with the writer helpers so length prefixes are consistent with the payload."],"exampleFix":"// before: decoding from a mis-offset buffer\nlet len = read_u32_le(buf)? as usize;\nlet v = buf[..len].to_vec();\n// after: keep offsets in sync with section header\nlet section = read_section_with_header(buf)?; // consumes length too\nlet msg = decode_prost_message::<Model>(&mut &section[..])?;","handlingStrategy":"validation","validationCode":"// ensure the declared length fits before decoding a prost message\nfn fits(buf: &[u8], declared: usize) -> bool { buf.len() >= declared }\n// e.g. after reading len: if !fits(&buf[..], len) { bail!(\"truncated protobuf section\"); }","typeGuard":"fn has_payload(buf: &[u8], len: usize) -> bool { buf.len() >= len }","tryCatchPattern":"match decode_section(&mut buf) {\n    Ok(model) => model,\n    Err(e) if e.to_string().contains(\"exceeds remaining buffer\") => {\n        log::error!(\"snapshot truncated or framing desynced at offset\");\n        return Err(anyhow!(\"snapshot protobuf section truncated\"));\n    }\n    Err(e) => return Err(e.into()),\n}","preventionTips":["Track section offsets carefully; a framing desync turns garbage bytes into bogus length prefixes.","Verify snapshot checksums before protobuf parsing.","Regenerate fixtures with the writer so length prefixes always match payloads."],"tags":["rust","backup","protobuf","snapshot-decoding","truncated-data"],"backgroundTag":"protobuf-unmarshal-failed","analyzedSha":"6469eb736d691e8e9b8a419a57edd6429ca77417","analyzedAt":"2026-09-11T21:06:21.487Z","contentChangedAt":"2026-09-11T21:06:21.487Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}