{"record":{"id":"3f5314198eeecce4","repo":"zeroclaw-labs/zeroclaw","slug":"email-oauth2-token-refresh-is-in-backoff-for-rema","errorCode":null,"errorMessage":"Email OAuth2 token refresh is in backoff for {remaining}s due to previous failures","messagePattern":"Email OAuth2 token refresh is in backoff for (.+?)s due to previous failures","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"crates/zeroclaw-providers/src/auth/mod.rs","lineNumber":547,"sourceCode":"        let refresh_lock = refresh_lock_for_profile(&profile_id);\n        let _guard = refresh_lock.lock().await;\n\n        // Re-load after acquiring lock to avoid duplicate refreshes.\n        let data = self.store.load().await?;\n        let Some(latest_profile) = data.profiles.get(&profile_id) else {\n            return Ok(None);\n        };\n        let Some(latest_tokens) = latest_profile.token_set.as_ref() else {\n            anyhow::bail!(\"Email OAuth2 profile is missing token set: {profile_id}\");\n        };\n        if !latest_tokens.is_expiring_within(Duration::from_secs(SKEW_SECS)) {\n            return Ok(Some(latest_tokens.access_token.clone()));\n        }\n\n        let refresh_token = latest_tokens.refresh_token.clone().unwrap_or(refresh_token);\n\n        if let Some(remaining) = refresh_backoff_remaining(&profile_id) {\n            anyhow::bail!(\n                \"Email OAuth2 token refresh is in backoff for {remaining}s due to previous failures\"\n            );\n        }\n\n        let mut refreshed = match refresh_email_access_token_with_retries(\n            &self.client,\n            token_url,\n            client_id,\n            &refresh_token,\n            scopes,\n        )\n        .await\n        {\n            Ok(tokens) => {\n                clear_refresh_backoff(&profile_id);\n                tokens\n            }\n            Err(err) => {","sourceCodeStart":529,"sourceCodeEnd":565,"githubUrl":"https://github.com/zeroclaw-labs/zeroclaw/blob/88bb9c8533fc57ed7a03e36ca7c9ed2bf8336dcc/crates/zeroclaw-providers/src/auth/mod.rs#L529-L565","documentation":"get_valid_email_oauth2_token bails when the profile is in a refresh backoff window: a previous refresh attempt failed with a retryable (transient) error, so set_refresh_backoff installed a deadline (OPENAI_REFRESH_FAILURE_BACKOFF_SECS = 10 seconds) and every call inside that window fails fast instead of hammering the token endpoint. The message reports how many seconds remain. Once the deadline passes, refresh_backoff_remaining returns None and normal refresh resumes; a later success calls clear_refresh_backoff.","triggerScenarios":"Calling get_valid_email_oauth2_token (directly or via imap_connect / get_oauth2_token) within ~10 seconds after a transient refresh failure such as a 5xx, network timeout, or temporarily_unavailable from the IdP token endpoint; repeated IMAP connects in a tight retry loop right after an outage.","commonSituations":"The IdP (Microsoft/Google) token endpoint is flaky or rate-limiting, a proxy drops connections, or a supervisor restarts the email channel loop faster than the 10s backoff, so every startup attempt lands inside the penalty window.","solutions":["Wait for the reported remaining seconds (max ~10s) and retry — the backoff self-clears when the deadline passes","Fix the underlying transient cause: network egress to the token_url, proxy/TLS issues, or IdP rate limiting","If the refresh token itself is dead you will see invalid_grant (no backoff is set for non-retryable errors) — re-run the email OAuth login instead of retrying","In channel restart loops, add a delay >= the backoff window before reconnecting"],"exampleFix":"// before: immediate reconnect loop\nloop { imap_connect(&cfg).await?; }\n\n// after: honor the backoff hinted by the error\nmatch auth.get_valid_email_oauth2_token(alias, None, url, id, &scopes).await {\n    Err(e) if e.to_string().contains(\"in backoff\") => {\n        tokio::time::sleep(Duration::from_secs(10)).await;\n    }\n    other => { other?; break; }\n}","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"let token = loop {\n    match auth.get_valid_email_oauth2_token(alias, None, url, id, &scopes).await {\n        Ok(t) => break t.context(\"no email profile configured\")?,\n        Err(e) => {\n            let msg = e.to_string();\n            if let Some(secs) = msg\n                .strip_prefix(\"Email OAuth2 token refresh is in backoff for \")\n                .and_then(|rest| rest.split('s').next())\n                .and_then(|s| s.trim().parse::<u64>().ok())\n            {\n                tokio::time::sleep(Duration::from_secs(secs.max(1))).await;\n                continue; // backoff elapsed, retry\n            }\n            return Err(e);\n        }\n    }\n};","preventionTips":["Space email-channel reconnect attempts by at least the 10s backoff window","Distinguish backoff (retryable) from invalid_grant (re-login required) before retrying","Keep token endpoint egress healthy so backoff rarely engages"],"tags":["oauth2","email","backoff","rate-limit","transient"],"backgroundTag":"oauth-refresh-backoff","analyzedSha":"88bb9c8533fc57ed7a03e36ca7c9ed2bf8336dcc","analyzedAt":"2026-08-23T01:07:41.857Z","schemaVersion":2},"datasetVersion":"2026-08-23T08:06:27.607Z"}