rustfs/rustfs · error

credential task join mutex poisoned

Error message

credential task join mutex poisoned

What it means

Panic-style guard in CredentialTaskHandle::shutdown: the mutex holding the renewal task's JoinHandle was poisoned by an earlier panic in another thread that held it. Shutdown expects the lock to be healthy so it can cancel and join the task; a poisoned lock means the handle's lifecycle state is untrustworthy and the process takes the panic rather than leaking the task silently.

Source

Thrown at crates/kms/src/backends/vault_credentials.rs:975

            }
        }
    }
}

/// Owner handle for a spawned renewal task.
///
/// Dropping the handle cancels the task, so hanging it off the service version
/// recycles the task on stop and reconfigure without explicit lifecycle calls.
pub(crate) struct CredentialTaskHandle {
    cancel: CancellationToken,
    join: std::sync::Mutex<Option<tokio::task::JoinHandle<()>>>,
}

impl CredentialTaskHandle {
    /// Cancel the renewal task and wait for it to exit.
    pub(crate) async fn shutdown(&self) {
        self.cancel.cancel();
        let join = self.join.lock().expect("credential task join mutex poisoned").take();
        if let Some(join) = join {
            let _ = join.await;
        }
    }
}

impl Drop for CredentialTaskHandle {
    fn drop(&mut self) {
        self.cancel.cancel();
    }
}

#[cfg(test)]
mod tests {
    use super::*;
    use crate::config::{DEFAULT_VAULT_KUBERNETES_MOUNT, REDACTED_SECRET};
    use std::sync::atomic::{AtomicBool, AtomicU32, Ordering};

View on GitHub (pinned to 35af688cd9)

Solutions

1. Fix the panic inside the renewal task that poisoned the lock
2. Recycle the credential task via the service version handle

When it happens

Trigger: Thrown at crates/kms/src/backends/vault_credentials.rs:975 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of rustfs/rustfs@35af688cd9 (2026-08-20). Data as JSON: /api/errors/20f2de9024fd2fab. Report an issue: GitHub.