{"record":{"id":"28748964e8076bab","repo":"tikv/tikv","slug":"failed-to-get-either-modify-lock-or-client","errorCode":null,"errorMessage":"failed to get either modify_lock or client","messagePattern":"failed to get either modify_lock or client","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"components/cloud/azure/src/azblob.rs","lineNumber":541,"sourceCode":"            let scopes = vec![&self.token_resource as &str];\n            let token =\n                self.token_cred.get_token(&scopes).await.map_err(|e| {\n                    io::Error::new(io::ErrorKind::InvalidInput, format!(\"{:?}\", &e))\n                })?;\n            let blob_service = BlobServiceClient::new(\n                self.account_name.clone(),\n                StorageCredentials::bearer_token(token.token.secret().to_string()),\n            );\n            let storage_client =\n                Arc::new(blob_service.container_client(self.container_name.clone()));\n\n            {\n                let mut token_response = self.token_cache.write().unwrap();\n                *token_response = Some((token, storage_client.clone()));\n            }\n            Ok(storage_client)\n        } else {\n            Err(io::Error::new(\n                io::ErrorKind::InvalidInput,\n                \"failed to get either modify_lock or client\",\n            ))\n        }\n    }\n}\n\nconst STORAGE_NAME: &str = \"azure\";\n\n#[derive(Clone)]\npub struct AzureStorage {\n    config: Config,\n    client_builder: Arc<dyn ContainerBuilder>,\n}\n\nimpl AzureStorage {\n    pub fn from_input(input: InputConfig) -> io::Result<Self> {\n        Self::new(Config::from_input(input)?)","sourceCodeStart":523,"sourceCodeEnd":559,"githubUrl":"https://github.com/tikv/tikv/blob/78aedc1c81ef3f7d8bacc6e9d09f56460f134937/components/cloud/azure/src/azblob.rs#L523-L559","documentation":"In get_client, the code attempts to acquire the modify_lock to build/refresh a token-authenticated BlobServiceClient; if the lock is poisoned (a previous holder panicked while holding the lock) or the lock request fails, neither a client nor the lock can be obtained and this InvalidInput error is returned.","triggerScenarios":"Calling get_client with Azure AD token credentials when self.modify_lock.write() fails — i.e. the RwLock is poisoned because another thread panicked while updating token_cache.","commonSituations":"A prior panic inside the token refresh critical section (e.g. unwrap on poisoned cache) leaving the RwLock poisoned; subsequent all calls to get_client then fail with this message on every retry.","solutions":["Fix the original panic that poisoned modify_lock — check logs for the earlier panic in the token refresh path.","Restart the affected TiKV/BR process to recreate the lock state.","Harden the critical section to avoid panics (no unwrap on token_cache entries)."],"exampleFix":"// before: panics propagate and poison the lock\nlet mut token_response = self.token_cache.write().unwrap();\n// after: recover or fail cleanly without poisoning on every call\nmatch self.modify_lock.write() {\n    Ok(mut guard) => { /* refresh */ },\n    Err(poisoned) => return Err(io::Error::new(io::ErrorKind::Other, \"token cache poisoned, restart required\")),\n}","handlingStrategy":"fallback","validationCode":null,"typeGuard":null,"tryCatchPattern":"// recover from poisoned lock instead of failing forever\nlet client = match self.modify_lock.write() {\n    Ok(g) => build_client(g),\n    Err(poisoned) => {\n        log::warn!(\"modify_lock poisoned; recovering\");\n        build_client(poisoned.into_inner())\n    }\n};","preventionTips":["Eliminate unwrap()/expect() inside the token-refresh critical section","Add panic recovery (catch_unwind) around cache updates if third-party code can panic","Restart the process promptly if you see this error — it repeats once the lock is poisoned"],"tags":["azure","lock","concurrency","poisoned-lock"],"backgroundTag":"lock-poisoned","analyzedSha":"78aedc1c81ef3f7d8bacc6e9d09f56460f134937","analyzedAt":"2026-09-03T23:31:32.398Z","contentChangedAt":"2026-09-03T23:31:32.398Z","schemaVersion":2},"datasetVersion":"2026-09-11T07:07:21.782Z"}