{"record":{"id":"095c44b65a013378","repo":"vectordotdev/vector","slug":"failed-to-acquire-write-lock-on-recency-map","errorCode":null,"errorMessage":"Failed to acquire write lock on recency map","messagePattern":"Failed to acquire write lock on recency map","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"lib/vector-core/src/metrics/recorder.rs","lineNumber":54,"sourceCode":"        self.registry.clear();\n    }\n\n    pub(super) fn set_expiry(\n        &self,\n        global_timeout: Option<Duration>,\n        expire_metrics_per_metric_set: Vec<(MetricKeyMatcher, Duration)>,\n    ) {\n        let recency = if global_timeout.is_none() && expire_metrics_per_metric_set.is_empty() {\n            None\n        } else {\n            Some(Recency::new(\n                Clock::new(),\n                MetricKindMask::ALL,\n                global_timeout,\n                expire_metrics_per_metric_set,\n            ))\n        };\n        *(self.recency.write()).expect(\"Failed to acquire write lock on recency map\") = recency;\n    }\n\n    pub(super) fn visit_metrics(&self) -> Vec<Metric> {\n        let timestamp = Utc::now();\n\n        let mut metrics = Vec::new();\n        let recency = self\n            .recency\n            .read()\n            .expect(\"Failed to acquire read lock on recency map\");\n        let recency = recency.as_ref();\n\n        for (key, counter) in self.registry.get_counter_handles() {\n            if recency\n                .is_none_or(|recency| recency.should_store_counter(&key, &counter, &self.registry))\n            {\n                // NOTE this will truncate if the value is greater than 2**52.\n                #[allow(clippy::cast_precision_loss)]","sourceCodeStart":36,"sourceCodeEnd":72,"githubUrl":"https://github.com/vectordotdev/vector/blob/bdb87aeaa4c4ff27c0ba643c1c77b21bf2ef4013/lib/vector-core/src/metrics/recorder.rs#L36-L72","documentation":"This is a Rust lock-poisoning panic. `set_expiry` replaces the recency map behind a `RwLock` using `self.recency.write()`. A `write()` call returns `Err` only when the lock is poisoned, i.e. another thread panicked while holding the write lock, and the code deliberately panics via `.expect(\"Failed to acquire write lock on recency map\")` instead of recovering.","triggerScenarios":"Calling `set_expiry` on the global metrics recorder when any prior thread/task panicked while holding the recency map's write lock (e.g. a panic inside the recency construction block or any earlier code that held the lock).","commonSituations":"A metrics-expiration update task panics once (bad timeout config, OOM-adjacent bug) and every subsequent `set_expiry` call then panics with this message; commonly surfaced in long-running Vector processes after an unrelated panic in the metrics subsystem.","solutions":["Find and fix the original panic that poisoned the lock: look earlier in the logs for the first panic in the metrics/recorder subsystem.","Upgrade or patch the recorder code to use a poisoning-tolerant lock (e.g. `parking_lot::RwLock` or recover from the `PoisonError` with `into_inner()`).","Restart the process; poisoning never clears at runtime, so the recorder cannot recover without a restart.","Audit config values passed into the recency-map construction (global_timeout, expire_metrics_per_metric_set) that may trigger the initial panic."],"exampleFix":"// before\n*(self.recency.write()).expect(\"Failed to acquire write lock on recency map\") = recency;\n// after\nlet mut guard = self\n    .recency\n    .write()\n    .unwrap_or_else(|poisoned| poisoned.into_inner());\n*guard = recency;","handlingStrategy":"try-catch","validationCode":"// Rust: cannot pre-validate poisoning; detect it before using the value\nlet poisoned = matches!(self.recency.try_write(), Err(std::sync::TryLockError::Poisoned(_)));","typeGuard":"fn is_poisoned<T>(lock: &std::sync::RwLock<T>) -> bool {\n    matches!(lock.try_read(), Err(std::sync::TryLockError::Poisoned(_)))\n}","tryCatchPattern":"// Rust panics are not catchable with try/catch; isolate the call site\nlet result = std::panic::catch_unwind(std::panic::AssertUnwindSafe(|| {\n    recorder.set_expiry(global_timeout, expire_metrics);\n}));\nif result.is_err() {\n    // recorder lock poisoned: restart metrics subsystem or process\n}","preventionTips":["Never panic while holding a lock; return Result from all code that mutates the recency map under the lock.","Prefer parking_lot::RwLock, which does not poison.","Monitor logs for any first panic in the metrics subsystem and treat it as a P1.","Wrap metric-expiration updates in catch_unwind at task boundaries so one panic does not poison global state."],"tags":["rust","locking","panic","poisoned-mutex"],"backgroundTag":"mutex-lock-poisoned","analyzedSha":"bdb87aeaa4c4ff27c0ba643c1c77b21bf2ef4013","analyzedAt":"2026-09-16T02:53:35.741Z","contentChangedAt":"2026-09-16T02:53:35.741Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}