{"record":{"id":"2657cc4662c07917","repo":"zeroclaw-labs/zeroclaw","slug":"send-failed-after-token-refresh-status-retry-sta","errorCode":null,"errorMessage":"send failed after token refresh: status={retry_status}, body={retry_response}","messagePattern":"send failed after token refresh: status=(.+?), body=(.+?)","errorType":"http","errorClass":"anyhow::Error","httpStatus":null,"severity":"error","filePath":"crates/zeroclaw-channels/src/lark.rs","lineNumber":2174,"sourceCode":"        Ok((status, parsed))\n    }\n\n    async fn send_json_with_token_refresh(\n        &self,\n        url: &str,\n        token: &mut String,\n        body: &serde_json::Value,\n        context: &str,\n    ) -> anyhow::Result<()> {\n        let (status, response) = self.send_text_once(url, token, body).await?;\n\n        if should_refresh_lark_tenant_token(status, &response) {\n            self.invalidate_token().await;\n            *token = self.get_tenant_access_token().await?;\n            let (retry_status, retry_response) = self.send_text_once(url, token, body).await?;\n\n            if should_refresh_lark_tenant_token(retry_status, &retry_response) {\n                anyhow::bail!(\n                    \"send failed after token refresh: status={retry_status}, body={retry_response}\"\n                );\n            }\n\n            ensure_lark_send_success(retry_status, &retry_response, context)?;\n        } else {\n            ensure_lark_send_success(status, &response, context)?;\n        }\n\n        Ok(())\n    }\n\n    async fn post_multipart_once(\n        &self,\n        url: &str,\n        token: &str,\n        form: Form,\n    ) -> anyhow::Result<(reqwest::StatusCode, serde_json::Value)> {","sourceCodeStart":2156,"sourceCodeEnd":2192,"githubUrl":"https://github.com/zeroclaw-labs/zeroclaw/blob/88bb9c8533fc57ed7a03e36ca7c9ed2bf8336dcc/crates/zeroclaw-channels/src/lark.rs#L2156-L2192","documentation":"send_text_once returned a refresh-indicating result (HTTP 401 or Lark code 99991663); the channel invalidated the cached token, fetched a new tenant_access_token, retried the send — and the retry again returned a refresh-indicating response. Two consecutive auth failures around a fresh token point to a credential or configuration defect, not normal expiry (tokens are cached with TTL-based refresh skew, default 7200s). Retry status and full body are included.","triggerScenarios":"app_secret rotated between the token fetch and its use, token endpoint and message endpoint pointed at different regions via api_base, clock skew or a poisoned token cache, or two channel instances running with different credentials for the same app.","commonSituations":"A secret rotation in the Lark console while the bot was running; config hot-reload swapping credentials mid-flight; duplicated channel processes (stale and new config) fighting over the same app.","solutions":["Read body= in the message: code 99991663 with a brand-new token almost always means wrong app credentials.","Re-verify app_id/app_secret against the console and restart so all instances share one canonical config.","Ensure only one process runs per app credential and that api_base is region-consistent.","If credentials are confirmed good, stop retrying immediately on this error (one refresh was already attempted) and cool down."],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":"fn is_repeat_stale_token(err: &anyhow::Error) -> bool {\n    let s = err.to_string();\n    s.contains(\"send failed after token refresh\")\n}","tryCatchPattern":"if is_repeat_stale_token(&e) {\n    // one refresh already failed: cool down, re-verify config, then retry once later\n    lark.invalidate_token().await;\n    cooldown(Duration::from_secs(60)).await;\n    return lark.send(msg).await;\n}","preventionTips":["Run exactly one channel instance per Lark app credential.","On secret rotation, update config and restart atomically.","Cool down on repeat auth failures instead of hot-looping sends."],"tags":["lark","send","token-refresh","authentication","retry-exhausted"],"backgroundTag":"stale-access-token","analyzedSha":"88bb9c8533fc57ed7a03e36ca7c9ed2bf8336dcc","analyzedAt":"2026-08-23T01:07:41.857Z","schemaVersion":2},"datasetVersion":"2026-08-23T08:06:27.607Z"}