risingwavelabs/risingwave · error · BackupError

Meta storage is not empty before being restored

Error message

Meta storage is not empty before being restored

What it means

`BackupError::NonemptyMetaStorage` is raised during restore when the target meta storage already contains data. Restoring into a non-empty meta store would overwrite or conflict with existing cluster metadata, so the operation refuses to proceed.

Source

Thrown at src/storage/backup/src/error.rs:54

        #[backtrace]
        #[source]
        BoxedError,
    ),
    #[error("Encoding error: {0}")]
    Encoding(
        #[backtrace]
        #[source]
        BoxedError,
    ),
    #[error("Decoding error: {0}")]
    Decoding(
        #[backtrace]
        #[source]
        BoxedError,
    ),
    #[error("Checksum mismatch: expected {expected}, found: {found}")]
    ChecksumMismatch { expected: u64, found: u64 },
    #[error("Meta storage is not empty before being restored")]
    NonemptyMetaStorage,
    #[error(transparent)]
    Other(
        #[from]
        #[backtrace]
        anyhow::Error,
    ),
}

impl From<std::io::Error> for BackupError {
    fn from(err: std::io::Error) -> Self {
        if err.kind() == std::io::ErrorKind::UnexpectedEof {
            Self::Decoding(anyhow::anyhow!("unexpected EOF while decoding meta snapshot").into())
        } else {
            Self::BackupStorage(err.into())
        }
    }
}

View on GitHub (pinned to 6469eb736d)

Solutions

1. Point the restore at a fresh, empty meta store (new etcd instance or cleaned data directory).
2. If intentional, explicitly wipe the target meta storage before restoring (after confirming no live cluster depends on it).
3. Restore into a separate environment and switch over, instead of restoring in place.

Example fix

// before: restore into existing meta store
./risedev d default && restore backup
// after: wipe meta store first
./risedev k && rm -rf .risingwave/data && ./risedev d default && restore backup

// Ensure target meta store is empty before restore
let keys = meta_store.list_prefix("").await?;
if !keys.is_empty() {
    return Err("refusing restore: target meta store is not empty".into());
}

match restore().await {
    Err(e @ BackupError::NonemptyMetaStorage) => {
        tracing::error!("target meta store not empty; wipe it or point restore at a fresh store");
        // abort; require explicit operator confirmation to wipe
    }
    other => other?,
}

When it happens

Trigger: Running a meta backup restore against a meta store that already has a cluster initialized (existing snapshots, workers, or metadata keys) instead of a fresh/empty meta store.

Common situations: Re-running a restore command without cleaning the meta store; pointing the restore at a live cluster's etcd; forgotten cleanup after a failed previous restore attempt.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• snapshot id {} not found
• {k} not found in system_parameter table
• MetaStorage error: {0}
• expect state_store
• invalid state_store from metadata snapshot: {}

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/09c974bc6c7b5011. Report an issue: GitHub.