FuelLabs/fuel-core · critical · anyhow::Error

The lock is poisoned: {}

Error message

The lock is poisoned: {}

What it means

MemoryStore guards each storage column with a std::sync::Mutex. If any thread panicked while holding a column lock, the mutex becomes poisoned and every later lock() returns Err; the write path _insert_changes maps that to 'The lock is poisoned'. The error is a symptom — the real fault is the earlier panic inside a critical section.

Source

Thrown at crates/fuel-core/src/state/in_memory/memory_store.rs:143

    ) -> impl Iterator<Item = KeyItem> + use<Description> {
        let lock = self.inner[column.as_usize()].lock().expect("poisoned");

        let collection: Vec<_> = keys_iterator(&lock, prefix, start, direction)
            .map(|key| key.to_vec())
            .collect();

        collection.into_iter().map(Ok)
    }

    fn _insert_changes(
        &self,
        conflicts_finder: &mut HashSet<(u32, ReferenceBytesKey)>,
        changes: Changes,
    ) -> DatabaseResult<()> {
        for (column, btree) in changes.into_iter() {
            let mut lock = self.inner[column as usize]
                .lock()
                .map_err(|e| anyhow::anyhow!("The lock is poisoned: {}", e))?;

            for (key, operation) in btree.into_iter() {
                if !conflicts_finder.insert((column, key.clone())) {
                    return Err(DatabaseError::ConflictingChanges {
                        column,
                        key: key.clone(),
                    })
                }

                match operation {
                    WriteOperation::Insert(value) => {
                        lock.insert(key, value);
                    }
                    WriteOperation::Remove => {
                        lock.remove(&key);
                    }
                }
            }

View on GitHub (pinned to b9d4d170da)

Solutions

1. Locate the original panic in logs or backtraces and fix it — the poisoned lock is only the aftermath.
2. Restart the process, or re-create the in-memory store in tests, since in-memory state cannot be recovered from a poison.
3. Move long-lived or production-like workloads to RocksDB, which has no mutex-poisoning mode.
4. Audit custom code that runs while holding the store lock for unwrap, expect, and indexing panics.

Example fix

// before: panic inside a path that holds the column lock
lock.insert(key, value);
let v = lock.get(&key).unwrap(); // panics -> poisons the mutex for every later writer

// after: no panicking operations while the lock is held
lock.insert(key, value);
let v = match lock.get(&key) { Some(v) => v, None => return Err(...) };

match db.commit_changes(changes) {
    Err(e) if e.to_string().contains("The lock is poisoned") => {
        // a thread panicked while holding a column lock: log, rebuild the in-memory store,
        // and hunt the original panic in earlier logs — do not retry on the same store
    }
    rest => rest,
}

When it happens

Trigger: Any panic in a thread holding a MemoryStore column lock (indexing bug, unwrap on bad data, failing assertion) — afterwards all writes through _insert_changes fail with this message.

Common situations: Parallel test suites sharing an in-memory database where one test panics; panics in custom storage mutation code paths; fault-injection experiments that abort mid-write.

Related errors

• epoch token lock poisoned: {}
• Not implemented yet
• cannot access epoch token, poisoned lock
• The historical view is not implemented for `MemoryStore`
• Missing code for contract: {id}

AI-assisted analysis of FuelLabs/fuel-core@b9d4d170da (2026-08-16). Data as JSON: /api/errors/ec917e446a5f8e1e. Report an issue: GitHub.