{"record":{"id":"64549f2b48a0fff7","repo":"risingwavelabs/risingwave","slug":"metadata-json-payload-length-exceeds-remaining","errorCode":null,"errorMessage":"metadata JSON payload length {} exceeds remaining buffer {}","messagePattern":"metadata JSON payload length (.+?) exceeds remaining buffer (.+?)","errorType":"exception","errorClass":"BackupError","httpStatus":null,"severity":"error","filePath":"src/storage/backup/src/meta_snapshot_v2.rs","lineNumber":454,"sourceCode":"                \"metadata snapshot is truncated while reading u64\"\n            )));\n        }\n        Ok(buf.get_u64_le())\n    }\n\n    fn read_len_prefix_from_slice(buf: &mut &[u8]) -> BackupResult<usize> {\n        match read_u32_le(buf)? {\n            0 => read_u64_le(buf)?.try_into().map_err(|_| {\n                BackupError::Other(anyhow!(\"metadata JSON payload length exceeds usize\"))\n            }),\n            len => Ok(len as usize),\n        }\n    }\n\n    fn skip_with_len_prefix_from_slice(buf: &mut &[u8]) -> BackupResult<()> {\n        let len = read_len_prefix_from_slice(buf)?;\n        if buf.remaining() < len {\n            return Err(BackupError::Other(anyhow!(\n                \"metadata JSON payload length {} exceeds remaining buffer {}\",\n                len,\n                buf.remaining()\n            )));\n        }\n        buf.advance(len);\n        Ok(())\n    }\n\n    fn skip_metadata_list_from_slice(buf: &mut &[u8]) -> BackupResult<()> {\n        let n = read_u32_le(buf)? as usize;\n        for _ in 0..n {\n            skip_with_len_prefix_from_slice(buf)?;\n        }\n        Ok(())\n    }\n\n    fn put_with_len_prefix(buf: &mut Vec<u8>, data: &impl Serialize) {","sourceCodeStart":436,"sourceCodeEnd":472,"githubUrl":"https://github.com/risingwavelabs/risingwave/blob/6469eb736d691e8e9b8a419a57edd6429ca77417/src/storage/backup/src/meta_snapshot_v2.rs#L436-L472","documentation":"This error is thrown when the declared length prefix of a metadata JSON payload is larger than the number of bytes left in the snapshot buffer, so skip_with_len_prefix_from_slice cannot advance past the payload. It means the snapshot payload is truncated or the length prefix was decoded from wrong bytes.","triggerScenarios":"skip_metadata_list_from_slice skips an entry whose length prefix exceeds buf.remaining() — a truncated buffer, an oversized/corrupt length field, or prior misaligned reads inflating the declared length.","commonSituations":"Restoring from an incomplete backup upload; parsing a buffer that was sliced at the wrong offset; corrupted snapshot files after disk or network failures.","solutions":["Compare the buffer length with the expected snapshot size in the backup manifest and re-fetch the backup if short","Fix upstream slice handling so reads/skips land on correct field boundaries before this call","Validate the snapshot checksum before parsing to detect corruption early"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"fn can_skip(buf: &[u8], prefix: &[u8]) -> bool {\n    let p = u32::from_le_bytes(prefix[..4].try_into().unwrap()) as usize;\n    p <= buf.len()\n}","typeGuard":null,"tryCatchPattern":"match skip_metadata_list_from_slice(&mut buf) {\n    Err(e) if format!(\"{e:#}\").contains(\"exceeds remaining buffer\") => {\n        // buffer truncated or misaligned: abort restore, verify checksum/size\n    }\n    other => other?,\n}","preventionTips":["Verify the full payload is loaded before parsing (compare against expected size)","Maintain explicit offset tracking instead of ad-hoc advances","Fail fast on checksum mismatch before walking metadata lists"],"tags":["rust","backup","truncation","serialization"],"backgroundTag":"unexpected-response-shape","analyzedSha":"6469eb736d691e8e9b8a419a57edd6429ca77417","analyzedAt":"2026-09-11T21:06:21.487Z","contentChangedAt":"2026-09-11T21:06:21.487Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}