rustfs/rustfs · critical · KmsError

Baseline version lost for key {key_id}: master key version r

Error message

Baseline version lost for key {key_id}: master key version records exist (oldest {oldest_version}) but the key record carries no baseline version, so data keys written before versioned rotation can no longer be resolved to the master key version that wrapped them. A node older than versioned rotation rewrote the key record and dropped the field. Finish upgrading every node, restore baseline_version to {oldest_version} on the key record, then retry

What it means

The key record carries master-key version records but no `baseline_version`, so data-key envelopes written before versioned rotation can no longer be mapped to the master key version that wrapped them. The stated cause is a node running code older than versioned rotation that rewrote the key record and dropped the field. Key listing marks such keys `Unreadable` (backends/mod.rs:299). `oldest_version` is exactly the baseline that was dropped, because version records start at the baseline the first rotation froze (error.rs:329-334).

Source

Thrown at crates/kms/src/error.rs:144

    KeyVersionNotFound { key_id: String, version: u32 },

    /// Backup/restore bundle contract violation; see [`crate::backup::BackupError`]
    #[error(transparent)]
    Backup(#[from] crate::backup::BackupError),

    /// Operation is not supported by the active KMS backend
    #[error("Operation '{operation}' is not supported by KMS backend '{backend}'")]
    UnsupportedCapability { backend: String, operation: String },

    /// Backend credentials expired or could not be refreshed in time; requests
    /// fail closed instead of being sent with credentials that may lapse mid-flight
    #[error("KMS credentials unavailable: {message}")]
    CredentialsUnavailable { message: String },

    /// Key has master key version records but no baseline version, so envelopes
    /// written before versioned rotation can no longer be resolved
    #[error(
        "Baseline version lost for key {key_id}: master key version records exist (oldest {oldest_version}) but the key record carries no baseline version, so data keys written before versioned rotation can no longer be resolved to the master key version that wrapped them. A node older than versioned rotation rewrote the key record and dropped the field. Finish upgrading every node, restore baseline_version to {oldest_version} on the key record, then retry"
    )]
    BaselineVersionLost { key_id: String, oldest_version: u32 },

    /// Configuration still points at the key, so its material must not be
    /// destroyed. Distinct from the generic invalid-operation errors so that
    /// callers can tell "this key is still wired into the deployment" apart
    /// from a malformed request and act on the listed references.
    #[error(
        "Key {key_id} is still referenced by configuration and its material must not be destroyed: {}. Remove or repoint the listed configuration, then retry",
        .references.join(", ")
    )]
    KeyStillReferenced { key_id: String, references: Vec<String> },

    /// The only available way to rewrap this envelope would pull the plaintext
    /// data key into the RustFS process. Refused rather than performed: the
    /// point of a backend-side rewrap is that the data key stays inside the
    /// backend, so silently falling back to unwrap-then-rewrap would hand back
    /// a correct envelope while quietly dropping the property that justified

View on GitHub (pinned to 35af688cd9)

Solutions

Finish upgrading every node to a version that understands versioned rotation so no old writer remains
Restore `baseline_version` on the key record to the `oldest_version` value the error reports
Retry the failing operation; verify the key now lists as readable again

Defensive patterns

Strategy: try-catch

Type guard

fn is_baseline_version_lost(e: &rustfs_kms::error::KmsError) -> bool {
    matches!(e, rustfs_kms::error::KmsError::BaselineVersionLost { .. })
}

Try / catch

if let Err(rustfs_kms::error::KmsError::BaselineVersionLost { key_id, oldest_version }) = op.await {
    page_operator(key_id, *oldest_version); // needs manual record repair
}

Prevention

Never run mixed versions across nodes where one version predates versioned rotation
Back up the Vault KV key record before downgrades or experiments
Alert on keys listing as Unreadable — that classification includes this condition

When it happens

Trigger: A mixed-version cluster where an old binary edits the Vault KV2 key record after a newer node performed versioned rotation; downgrading a node below versioned-rotation support and letting it touch the key; hand-editing the key record without the `baseline_version` field.

Common situations: Rolling upgrade paused mid-way with KMS writes happening on both versions; a rollback of one node while others already rotated keys; restore of a key record from an old backup.

Related errors

AI-assisted analysis of rustfs/rustfs@35af688cd9 (2026-08-20). Data as JSON: /api/errors/7b937385ab76d024. Report an issue: GitHub.