{"record":{"id":"248945da2ba397e4","repo":"risingwavelabs/risingwave","slug":"metadata-json-payload-length-exceeds-usize","errorCode":null,"errorMessage":"metadata JSON payload length exceeds usize","messagePattern":"metadata JSON payload length exceeds usize","errorType":"exception","errorClass":"BackupError","httpStatus":null,"severity":"error","filePath":"src/storage/backup/src/meta_snapshot_v2.rs","lineNumber":339,"sourceCode":"\nasync fn skip_metadata_list(reader: &mut SnapshotPayloadReader) -> BackupResult<()> {\n    let n = reader.read_u32_le().await? as usize;\n    for _ in 0..n {\n        skip_with_len_prefix(reader).await?;\n    }\n    Ok(())\n}\n\nasync fn skip_with_len_prefix(reader: &mut SnapshotPayloadReader) -> BackupResult<()> {\n    let len = read_len_prefix(reader).await?;\n    reader.skip_exact(len).await\n}\n\nasync fn read_len_prefix(reader: &mut SnapshotPayloadReader) -> BackupResult<usize> {\n    match reader.read_u32_le().await? {\n        0 => {\n            reader.read_u64_le().await?.try_into().map_err(|_| {\n                BackupError::Other(anyhow!(\"metadata JSON payload length exceeds usize\"))\n            })\n        }\n        len => Ok(len as usize),\n    }\n}\n\nasync fn write_with_len_prefix<T: Serialize>(\n    writer: &mut SnapshotPayloadWriter,\n    data: &T,\n) -> BackupResult<()> {\n    let b = serde_json::to_vec(data)?;\n    // Any valid JSON value serialized by serde_json is non-empty, such as `null`, `{}`, or `[]`.\n    assert!(!b.is_empty());\n    let len_prefix_len = if u32::try_from(b.len()).is_ok() {\n        size_of::<u32>()\n    } else {\n        size_of::<u32>() + size_of::<u64>()\n    };","sourceCodeStart":321,"sourceCodeEnd":357,"githubUrl":"https://github.com/risingwavelabs/risingwave/blob/6469eb736d691e8e9b8a419a57edd6429ca77417/src/storage/backup/src/meta_snapshot_v2.rs#L321-L357","documentation":"This error is thrown when reading the length prefix of a metadata JSON payload in a v2 metadata snapshot. The wire format stores the length as a u32 when it fits, or as a marker 0 followed by a full u64 for larger payloads. When the u64 form is used, its value must fit in usize; otherwise the payload length is meaningless on this platform and the snapshot is treated as corrupt.","triggerScenarios":"read_len_prefix reads a u32 length-prefix of 0, then reads a u64 whose value exceeds usize::MAX (e.g. a corrupted or maliciously crafted snapshot file, or a truncated/misaligned stream where unrelated bytes are interpreted as a huge u64 length).","commonSituations":"Restoring from a corrupted or partially written backup snapshot file; byte-offset misalignment after earlier parse errors on 32-bit targets where u64 lengths never fit; hand-edited or bit-rotted snapshot payloads.","solutions":["Verify the snapshot file integrity (checksum) against the backup manifest and re-upload/re-create the backup if it is corrupted","Ensure the payload reader is correctly positioned — a misaligned stream can make garbage bytes decode as a huge length; fix upstream parsing so offsets stay aligned","If running on a 32-bit platform, restore on a 64-bit host where realistic metadata sizes always fit in usize"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"fn validate_len_prefix(bytes: &[u8]) -> Result<usize, String> {\n    if bytes.len() < 4 { return Err(\"buffer too short for u32 length\".into()); }\n    let prefix = u32::from_le_bytes(bytes[..4].try_into().unwrap());\n    if prefix == 0 {\n        if bytes.len() < 12 { return Err(\"buffer too short for u64 length\".into()); }\n        let len = u64::from_le_bytes(bytes[4..12].try_into().unwrap());\n        usize::try_from(len).map_err(|_| \"u64 length exceeds usize\".to_string())\n    } else {\n        Ok(prefix as usize)\n    }\n}","typeGuard":"fn fits_usize(len: u64) -> bool { usize::try_from(len).is_ok() }","tryCatchPattern":null,"preventionTips":["Always checksum-verify snapshot files before parsing","Keep a running byte-offset invariant in payload parsers and assert it after each field","Test snapshot parsing on 32-bit targets if you support them"],"tags":["rust","backup","serialization","corruption"],"backgroundTag":"value-out-of-range","analyzedSha":"6469eb736d691e8e9b8a419a57edd6429ca77417","analyzedAt":"2026-09-11T21:06:21.487Z","contentChangedAt":"2026-09-11T21:06:21.487Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}