risingwavelabs/risingwave · error · BackupError

metadata JSON payload length

Error message

metadata JSON payload length {} exceeds remaining buffer {}

What it means

This error is thrown when the declared length prefix of a metadata JSON payload is larger than the number of bytes left in the snapshot buffer, so skip_with_len_prefix_from_slice cannot advance past the payload. It means the snapshot payload is truncated or the length prefix was decoded from wrong bytes.

Solutions

1. Compare the buffer length with the expected snapshot size in the backup manifest and re-fetch the backup if short
2. Fix upstream slice handling so reads/skips land on correct field boundaries before this call
3. Validate the snapshot checksum before parsing to detect corruption early

fn can_skip(buf: &[u8], prefix: &[u8]) -> bool {
    let p = u32::from_le_bytes(prefix[..4].try_into().unwrap()) as usize;
    p <= buf.len()
}

match skip_metadata_list_from_slice(&mut buf) {
    Err(e) if format!("{e:#}").contains("exceeds remaining buffer") => {
        // buffer truncated or misaligned: abort restore, verify checksum/size
    }
    other => other?,
}

When it happens

Trigger: skip_metadata_list_from_slice skips an entry whose length prefix exceeds buf.remaining() — a truncated buffer, an oversized/corrupt length field, or prior misaligned reads inflating the declared length.

Common situations: Restoring from an incomplete backup upload; parsing a buffer that was sliced at the wrong offset; corrupted snapshot files after disk or network failures.

Related errors

• metadata snapshot is truncated while reading u64
• Decoding error
• Encoding error
• metadata JSON payload length exceeds usize
• BackupStorage error

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/64549f2b48a0fff7. Report an issue: GitHub.

Appendix: source

Thrown at src/storage/backup/src/meta_snapshot_v2.rs:454

                "metadata snapshot is truncated while reading u64"
            )));
        }
        Ok(buf.get_u64_le())
    }

    fn read_len_prefix_from_slice(buf: &mut &[u8]) -> BackupResult<usize> {
        match read_u32_le(buf)? {
            0 => read_u64_le(buf)?.try_into().map_err(|_| {
                BackupError::Other(anyhow!("metadata JSON payload length exceeds usize"))
            }),
            len => Ok(len as usize),
        }
    }

    fn skip_with_len_prefix_from_slice(buf: &mut &[u8]) -> BackupResult<()> {
        let len = read_len_prefix_from_slice(buf)?;
        if buf.remaining() < len {
            return Err(BackupError::Other(anyhow!(
                "metadata JSON payload length {} exceeds remaining buffer {}",
                len,
                buf.remaining()
            )));
        }
        buf.advance(len);
        Ok(())
    }

    fn skip_metadata_list_from_slice(buf: &mut &[u8]) -> BackupResult<()> {
        let n = read_u32_le(buf)? as usize;
        for _ in 0..n {
            skip_with_len_prefix_from_slice(buf)?;
        }
        Ok(())
    }

    fn put_with_len_prefix(buf: &mut Vec<u8>, data: &impl Serialize) {

View on GitHub (pinned to 6469eb736d)