{"record":{"id":"ec917e446a5f8e1e","repo":"FuelLabs/fuel-core","slug":"the-lock-is-poisoned","errorCode":null,"errorMessage":"The lock is poisoned: {}","messagePattern":"The lock is poisoned: (.+?)","errorType":"exception","errorClass":"anyhow::Error","httpStatus":null,"severity":"critical","filePath":"crates/fuel-core/src/state/in_memory/memory_store.rs","lineNumber":143,"sourceCode":"    ) -> impl Iterator<Item = KeyItem> + use<Description> {\n        let lock = self.inner[column.as_usize()].lock().expect(\"poisoned\");\n\n        let collection: Vec<_> = keys_iterator(&lock, prefix, start, direction)\n            .map(|key| key.to_vec())\n            .collect();\n\n        collection.into_iter().map(Ok)\n    }\n\n    fn _insert_changes(\n        &self,\n        conflicts_finder: &mut HashSet<(u32, ReferenceBytesKey)>,\n        changes: Changes,\n    ) -> DatabaseResult<()> {\n        for (column, btree) in changes.into_iter() {\n            let mut lock = self.inner[column as usize]\n                .lock()\n                .map_err(|e| anyhow::anyhow!(\"The lock is poisoned: {}\", e))?;\n\n            for (key, operation) in btree.into_iter() {\n                if !conflicts_finder.insert((column, key.clone())) {\n                    return Err(DatabaseError::ConflictingChanges {\n                        column,\n                        key: key.clone(),\n                    })\n                }\n\n                match operation {\n                    WriteOperation::Insert(value) => {\n                        lock.insert(key, value);\n                    }\n                    WriteOperation::Remove => {\n                        lock.remove(&key);\n                    }\n                }\n            }","sourceCodeStart":125,"sourceCodeEnd":161,"githubUrl":"https://github.com/FuelLabs/fuel-core/blob/b9d4d170da3a31c9ace5f963d633b326348e0d42/crates/fuel-core/src/state/in_memory/memory_store.rs#L125-L161","documentation":"MemoryStore guards each storage column with a std::sync::Mutex. If any thread panicked while holding a column lock, the mutex becomes poisoned and every later lock() returns Err; the write path _insert_changes maps that to 'The lock is poisoned'. The error is a symptom — the real fault is the earlier panic inside a critical section.","triggerScenarios":"Any panic in a thread holding a MemoryStore column lock (indexing bug, unwrap on bad data, failing assertion) — afterwards all writes through _insert_changes fail with this message.","commonSituations":"Parallel test suites sharing an in-memory database where one test panics; panics in custom storage mutation code paths; fault-injection experiments that abort mid-write.","solutions":["Locate the original panic in logs or backtraces and fix it — the poisoned lock is only the aftermath.","Restart the process, or re-create the in-memory store in tests, since in-memory state cannot be recovered from a poison.","Move long-lived or production-like workloads to RocksDB, which has no mutex-poisoning mode.","Audit custom code that runs while holding the store lock for unwrap, expect, and indexing panics."],"exampleFix":"// before: panic inside a path that holds the column lock\nlock.insert(key, value);\nlet v = lock.get(&key).unwrap(); // panics -> poisons the mutex for every later writer\n\n// after: no panicking operations while the lock is held\nlock.insert(key, value);\nlet v = match lock.get(&key) { Some(v) => v, None => return Err(...) };","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"match db.commit_changes(changes) {\n    Err(e) if e.to_string().contains(\"The lock is poisoned\") => {\n        // a thread panicked while holding a column lock: log, rebuild the in-memory store,\n        // and hunt the original panic in earlier logs — do not retry on the same store\n    }\n    rest => rest,\n}","preventionTips":["Treat any panic in storage-touching threads as fatal for an in-memory db; fail fast and restart.","Avoid unwrap/expect and direct indexing in code paths that hold MemoryStore locks.","Isolate tests so one panicking test cannot poison a shared store.","Prefer RocksDB for long-lived or production-like deployments."],"tags":["concurrency","panic","in-memory","mutex","fuel-core","rust"],"backgroundTag":null,"analyzedSha":"b9d4d170da3a31c9ace5f963d633b326348e0d42","analyzedAt":"2026-08-16T08:56:42.692Z","schemaVersion":2},"datasetVersion":"2026-08-16T13:17:31.715Z"}