What causes "not poisoned" in influxdata/influxdb?

Find the original panic in the logs — it came from code holding the write side of this entry's lock (typically V::in_use) — and fix V so it cannot panic.

influxdata/influxdb · error

not poisoned

Error message

not poisoned

What it means

S3FifoEntry stores its value in a std RwLock; value() takes a read guard with .expect("not poisoned"). The RwLock is poisoned when some thread panicked while holding a WRITE guard on this same entry's value — most commonly a panic inside V::in_use() (which takes self.value.write()) or any internal mutation path. From then on, every read of that entry's value panics with this message.

Source

Thrown at core/object_store_mem_cache/src/cache_system/s3_fifo_cache/s3_fifo.rs:40

/// Entry within [`S3Fifo`].
#[derive(Debug)]
pub struct S3FifoEntry<K, V>
where
    K: ?Sized,
{
    key: Arc<K>,
    value: RwLock<V>,
    generation: u64,
    freq: AtomicU8,
}

impl<K, V> S3FifoEntry<K, V>
where
    K: ?Sized,
{
    pub(crate) fn value(&self) -> RwLockReadGuard<'_, V> {
        self.value.read().expect("not poisoned")
    }
}

impl<K, V> HasSize for S3FifoEntry<K, V>
where
    K: HasSize + ?Sized,
    V: HasSize,
{
    fn size(&self) -> usize {
        let Self {
            key,
            value,
            generation: _,
            freq: _,
        } = self;
        key.size() + value.read().expect("not poisoned").size()
    }
}

View on GitHub (pinned to d28e26e048)

Solutions

Find the original panic in the logs — it came from code holding the write side of this entry's lock (typically V::in_use) — and fix V so it cannot panic.
Restart (or evict/re-create the cache) to drop the poisoned entry; a poisoned RwLock never recovers.
Audit custom V implementations for unwraps/expects/indexing that can panic on malformed cached data.
Library-level option: change value() to recover guards via unwrap_or_else(|e| e.into_inner()), accepting possibly-inconsistent V state, or use parking_lot RwLock (no poisoning).

Example fix

// before
pub(crate) fn value(&self) -> RwLockReadGuard<'_, V> {
    self.value.read().expect("not poisoned")
}

// after
pub(crate) fn value(&self) -> RwLockReadGuard<'_, V> {
    self.value.read().unwrap_or_else(|e| e.into_inner())
}

Defensive patterns

Strategy: fallback

Prevention

Make V::in_use and any value-mutation code panic-free — they hold the write side and are the poison source.
Audit cached value types for unwrap/expect/index/ division that can panic on malformed data.
Run integration tests that force eviction and concurrent reads to surface lock poisoning early.
Consider catch_unwind around request handling that reads cached values, degrading to a cache miss instead of a panic.

When it happens

Trigger: Cache.get()/read paths on an entry whose value RwLock was poisoned earlier, i.e. after a panic inside a write-lock holder such as in_use() (s3_fifo.rs:69), entry_likely_in_use (s3_fifo.rs:82), or any code mutating the cached value under the write guard.

Common situations: A cached value type V whose in_use()/update logic panics once (bad data, index math, unwrap inside V); afterwards every reader of that key panics, turning one bad entry into repeated panics; crates built with panic=abort abort the process at the first write-side panic instead.

Related errors

AI-assisted analysis of influxdata/influxdb@d28e26e048 (2026-08-16). Data as JSON: /api/errors/8cc2e2e7edd225e0. Report an issue: GitHub.