{"record":{"id":"af18d086bfc5060e","repo":"firecracker-microvm/firecracker","slug":"poisoned-lock-af18d0","errorCode":null,"errorMessage":"Poisoned lock","messagePattern":"Poisoned lock","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"src/vmm/src/lib.rs","lineNumber":405,"sourceCode":"                VirtioDeviceType::Vsock => {\n                    if let Some(v) = device.as_any().downcast_ref::<Vsock<VsockUnixBackend>>() {\n                        vsock = Some(VsockDeviceConfig::from(v));\n                    }\n                }\n                VirtioDeviceType::Rng => {\n                    if let Some(e) = device.as_any().downcast_ref::<Entropy>() {\n                        entropy = Some(EntropyDeviceConfig::from(e));\n                    }\n                }\n                VirtioDeviceType::Mem => {\n                    if let Some(m) = device.as_any().downcast_ref::<VirtioMem>() {\n                        memory_hotplug = Some(MemoryHotplugConfig::from(m));\n                    }\n                }\n            });\n\n        let mmds_config = mmds_ref.map(|mmds| {\n            let mmds = mmds.lock().expect(\"Poisoned lock\");\n            MmdsConfig {\n                version: mmds.version(),\n                ipv4_address: mmds_ipv4_address,\n                network_interfaces: net_with_mmds,\n                imds_compat: mmds.imds_compat(),\n            }\n        });\n\n        // This must match the From<&VmResources> for VmmConfig implementation\n        // in resources.rs which is used to retrieve the config before the VM\n        // is started.\n        VmmConfig {\n            balloon,\n            drives: block,\n            boot_source: self.boot_source_config.clone(),\n            cpu_config: None,\n            logger: None,\n            machine_config: Some(self.machine_config.clone()),","sourceCodeStart":387,"sourceCodeEnd":423,"githubUrl":"https://github.com/firecracker-microvm/firecracker/blob/0a745def42ddf4cc2a744d79a08a27ff50b5d27a/src/vmm/src/lib.rs#L387-L423","documentation":"Panic in `Vmm` config export (lib.rs:405): `mmds.lock().expect(\"Poisoned lock\")` while building `MmdsConfig` from the MMDS service handle. The Mutex<VecDeque/Mmds> is poisoned because some thread panicked while holding it — every later `lock()` returns Err(PoisonError) and this serialization path aborts instead of degrading. The original panic is the true root cause; this is the follow-on failure.","triggerScenarios":"Any earlier panic in a thread holding the MMDS lock (e.g. a request handler unwinding inside `mmds.lock()`), followed by a config dump / DescribeInstances-style API call that serializes VmmConfig including MmdsConfig.","commonSituations":"Firecracker API queries the microvm configuration after a background MMDS/HTTP thread crashed; long-lived microvms where a poisoned lock surfaces only on the first later MMDS interaction; tests that intentionally panic inside MMDS handlers and then read config.","solutions":["Find and fix the FIRST panic that poisoned the lock (check stderr/log for the earlier panic message)","Make this read path poison-tolerant: `mmds.lock().unwrap_or_else(|e| e.into_inner())` — MMDS state is still readable after a poisoned unlock","Avoid holding the MMDS lock across code that can panic; scope guards tightly around data access only"],"exampleFix":"// before\nlet mmds = mmds.lock().expect(\"Poisoned lock\");\n\n// after\nlet mmds = mmds.lock().unwrap_or_else(|poisoned| poisoned.into_inner());","handlingStrategy":"fallback","validationCode":"// Before serializing config, detect poisoning cheaply\nif let Ok(mmds) = mmds.try_lock() {\n    // healthy path: build MmdsConfig\n} else if mmds.is_poisoned() {\n    // recover or report degraded config\n}","typeGuard":null,"tryCatchPattern":"// Read through the poison: MMDS content is still consistent enough for a config dump\nlet mmds = mmds.lock().unwrap_or_else(|p| p.into_inner());\nlet cfg = MmdsConfig { version: mmds.version(), .. };","preventionTips":["Scope MMDS lock guards to data access only; never across request parsing or unwraps","Install a panic hook that logs which thread poisoned which lock","Surface poison state via a health endpoint so config queries can report degraded instead of aborting"],"tags":["rust","panic","mutex","poisoned-lock","mmds","metadata","firecracker"],"backgroundTag":"mutex-poisoned","analyzedSha":"0a745def42ddf4cc2a744d79a08a27ff50b5d27a","analyzedAt":"2026-08-19T05:27:02.517Z","contentChangedAt":"2026-08-19T05:27:02.517Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}