firecracker-microvm/firecracker · error

error unlocking vmm

Error message

error unlocking vmm

What it means

Panic in `FirecrackerTarget::shutdown_vmm` (target.rs:276): `self.vmm.lock().expect("error unlocking vmm")` fires when the Vmm Mutex is poisoned — i.e. some other thread panicked while holding the lock, leaving it poisoned; every subsequent `lock()` returns Err(PoisonError). This is a secondary failure: the real fault is whatever panicked earlier while holding the Vmm lock.

Source

Thrown at src/vmm/src/gdb/target.rs:276

        }

        self.paused_vcpu = None;

        Ok(())
    }

    /// Resets all Vcpus to their base state
    fn reset_all_vcpu_states(&mut self) {
        for value in self.vcpu_state.iter_mut() {
            value.reset_vcpu_state();
        }
    }

    /// Shuts down the VMM
    pub fn shutdown_vmm(&self) {
        self.vmm
            .lock()
            .expect("error unlocking vmm")
            .stop(FcExitCode::Ok)
    }

    /// Pauses the requested Vcpu
    pub fn pause_vcpu(&mut self, tid: Tid) -> Result<(), GdbTargetError> {
        let vcpu_state = &mut self.vcpu_state[tid_to_vcpuid(tid)];

        if vcpu_state.paused {
            info!("Attempted to pause a vcpu already paused.");
            // Pausing an already paused vcpu is not considered an error case from GDB
            return Ok(());
        }

        let vmm = self.vmm.lock()?;
        let kvm_vm = vmm.vm.as_kvm().expect("GDB requires KVM");
        let mut handles = kvm_vm.vcpus_handles();
        let cpu_handle = &mut handles[tid_to_vcpuid(tid)];

View on GitHub (pinned to 0a745def42)

Solutions

1. Inspect logs/stderr for the FIRST panic — fixing that removes the poisoning; this expect is only a symptom
2. Make shutdown resilient: `self.vmm.lock().unwrap_or_else(|e| e.into_inner()).stop(FcExitCode::Ok)` since stopping a poisoned VMM is still correct
3. Audit code that holds the Vmm lock across fallible operations and move panics out of the critical section

Example fix

// before
self.vmm
    .lock()
    .expect("error unlocking vmm")
    .stop(FcExitCode::Ok)

// after
self.vmm
    .lock()
    .unwrap_or_else(|poisoned| poisoned.into_inner())
    .stop(FcExitCode::Ok)

// Nothing to validate pre-call; instead ensure no panic ever holds the Vmm lock.
// Audit every `vmm.lock()` critical section for unwrap/expect/index panics.

// Treat a poisoned VMM as still stoppable — stop() is the terminal action anyway
let guard = self.vmm.lock().unwrap_or_else(|p| p.into_inner());
guard.stop(FcExitCode::Ok);

When it happens

Trigger: A GDB 'kill' command (or detach path that shuts down) calls shutdown_vmm after another thread already panicked inside `vmm.lock()` — e.g. one of the other `expect("GDB requires KVM")` sites or a vcpu error path unwinding through a lock guard.

Common situations: Any earlier panic anywhere in the process while the Vmm mutex was held (device error, KVM ioctl failure, the as_kvm expects) followed by the GDB session issuing shutdown; long-running firecracker processes where a background thread hit a corner-case panic.

Related errors

• Poisoned lock
• i8042 lock was poisoned
• GDB requires KVM
• Error getting initial gdb event
• Error converting cpu id to Tid

AI-assisted analysis of firecracker-microvm/firecracker@0a745def42 (2026-08-19). Data as JSON: /api/errors/746689da4610eed4. Report an issue: GitHub.