risingwavelabs/risingwave · error · BackupError

metadata snapshot is truncated while reading u64

Error message

metadata snapshot is truncated while reading u64

What it means

This error indicates the snapshot byte buffer had fewer than 8 bytes remaining when a little-endian u64 was required. It is a truncation guard used by the slice-based snapshot reader, signaling that the metadata snapshot payload ends mid-field and cannot be parsed.

Source

Thrown at src/storage/backup/src/meta_snapshot_v2.rs:435

        let uploader = store.streaming_upload(path).await.unwrap();
        snapshot.encode_to_uploader(uploader).await.unwrap();
    }

    async fn snapshot_reader(store: &ObjectStoreImpl, path: &str) -> SnapshotPayloadReader {
        SnapshotPayloadReader::new(ObjectDataStreamReader::new(
            store.streaming_read(path, ..).await.unwrap().into_stream(),
        ))
    }

    fn append_checksum(mut payload: Vec<u8>) -> Vec<u8> {
        let checksum = crate::xxhash64_checksum(&payload);
        payload.put_u64_le(checksum);
        payload
    }

    fn read_u64_le(buf: &mut &[u8]) -> BackupResult<u64> {
        if buf.remaining() < size_of::<u64>() {
            return Err(BackupError::Other(anyhow!(
                "metadata snapshot is truncated while reading u64"
            )));
        }
        Ok(buf.get_u64_le())
    }

    fn read_len_prefix_from_slice(buf: &mut &[u8]) -> BackupResult<usize> {
        match read_u32_le(buf)? {
            0 => read_u64_le(buf)?.try_into().map_err(|_| {
                BackupError::Other(anyhow!("metadata JSON payload length exceeds usize"))
            }),
            len => Ok(len as usize),
        }
    }

    fn skip_with_len_prefix_from_slice(buf: &mut &[u8]) -> BackupResult<()> {
        let len = read_len_prefix_from_slice(buf)?;
        if buf.remaining() < len {

View on GitHub (pinned to 6469eb736d)

Solutions

1. Check the snapshot file size against the size recorded in the backup manifest; re-transfer or re-create the backup if truncated
2. Confirm the caller passes the complete payload slice, not a partial read
3. Add/treat this as a corruption signal: fail the restore early instead of retrying parsing

fn has_u64(buf: &[u8]) -> bool { buf.remaining() >= std::mem::size_of::<u64>() }

match parse_snapshot(buf) {
    Err(BackupError::Other(e)) if format!("{e:#}").contains("truncated") => {
        // treat as corrupted/truncated snapshot: fail restore, prompt re-backup
    }
    other => other?,
}

When it happens

Trigger: read_u64_le is called on a &[u8] whose remaining() < 8 — e.g. read_len_prefix_from_slice encountering the 0 marker for u64-length but the buffer ends, or reading the trailing checksum of a truncated metadata snapshot.

Common situations: Restoring a metadata snapshot that was cut short by a failed upload, disk-full during backup write, or manually truncated file; passing a wrong/short byte slice to the parser.

Related errors

• metadata JSON payload length {} exceeds remaining buffer {}
• Encoding error: {0}
• Decoding error: {0}
• metadata JSON payload length exceeds usize
• iceberg sink metadata should have schema_id

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/5f05c9e5d0b8dc74. Report an issue: GitHub.