Email OAuth2 token refresh is in backoff for {remaining}s du

Error message

Email OAuth2 token refresh is in backoff for {remaining}s due to previous failures

What it means

get_valid_email_oauth2_token bails when the profile is in a refresh backoff window: a previous refresh attempt failed with a retryable (transient) error, so set_refresh_backoff installed a deadline (OPENAI_REFRESH_FAILURE_BACKOFF_SECS = 10 seconds) and every call inside that window fails fast instead of hammering the token endpoint. The message reports how many seconds remain. Once the deadline passes, refresh_backoff_remaining returns None and normal refresh resumes; a later success calls clear_refresh_backoff.

Source

Thrown at crates/zeroclaw-providers/src/auth/mod.rs:547

        let refresh_lock = refresh_lock_for_profile(&profile_id);
        let _guard = refresh_lock.lock().await;

        // Re-load after acquiring lock to avoid duplicate refreshes.
        let data = self.store.load().await?;
        let Some(latest_profile) = data.profiles.get(&profile_id) else {
            return Ok(None);
        };
        let Some(latest_tokens) = latest_profile.token_set.as_ref() else {
            anyhow::bail!("Email OAuth2 profile is missing token set: {profile_id}");
        };
        if !latest_tokens.is_expiring_within(Duration::from_secs(SKEW_SECS)) {
            return Ok(Some(latest_tokens.access_token.clone()));
        }

        let refresh_token = latest_tokens.refresh_token.clone().unwrap_or(refresh_token);

        if let Some(remaining) = refresh_backoff_remaining(&profile_id) {
            anyhow::bail!(
                "Email OAuth2 token refresh is in backoff for {remaining}s due to previous failures"
            );
        }

        let mut refreshed = match refresh_email_access_token_with_retries(
            &self.client,
            token_url,
            client_id,
            &refresh_token,
            scopes,
        )
        .await
        {
            Ok(tokens) => {
                clear_refresh_backoff(&profile_id);
                tokens
            }
            Err(err) => {

View on GitHub (pinned to 88bb9c8533)

Solutions

Wait for the reported remaining seconds (max ~10s) and retry — the backoff self-clears when the deadline passes
Fix the underlying transient cause: network egress to the token_url, proxy/TLS issues, or IdP rate limiting
If the refresh token itself is dead you will see invalid_grant (no backoff is set for non-retryable errors) — re-run the email OAuth login instead of retrying
In channel restart loops, add a delay >= the backoff window before reconnecting

Example fix

// before: immediate reconnect loop
loop { imap_connect(&cfg).await?; }

// after: honor the backoff hinted by the error
match auth.get_valid_email_oauth2_token(alias, None, url, id, &scopes).await {
    Err(e) if e.to_string().contains("in backoff") => {
        tokio::time::sleep(Duration::from_secs(10)).await;
    }
    other => { other?; break; }
}

Defensive patterns

Strategy: retry

Try / catch

let token = loop {
    match auth.get_valid_email_oauth2_token(alias, None, url, id, &scopes).await {
        Ok(t) => break t.context("no email profile configured")?,
        Err(e) => {
            let msg = e.to_string();
            if let Some(secs) = msg
                .strip_prefix("Email OAuth2 token refresh is in backoff for ")
                .and_then(|rest| rest.split('s').next())
                .and_then(|s| s.trim().parse::<u64>().ok())
            {
                tokio::time::sleep(Duration::from_secs(secs.max(1))).await;
                continue; // backoff elapsed, retry
            }
            return Err(e);
        }
    }
};

Prevention

Space email-channel reconnect attempts by at least the 10s backoff window
Distinguish backoff (retryable) from invalid_grant (re-login required) before retrying
Keep token endpoint egress healthy so backoff rarely engages

When it happens

Trigger: Calling get_valid_email_oauth2_token (directly or via imap_connect / get_oauth2_token) within ~10 seconds after a transient refresh failure such as a 5xx, network timeout, or temporarily_unavailable from the IdP token endpoint; repeated IMAP connects in a tight retry loop right after an outage.

Common situations: The IdP (Microsoft/Google) token endpoint is flaky or rate-limiting, a proxy drops connections, or a supervisor restarts the email channel loop faster than the 10s backoff, so every startup attempt lands inside the penalty window.

Related errors

AI-assisted analysis of zeroclaw-labs/zeroclaw@88bb9c8533 (2026-08-23). Data as JSON: /api/errors/3f5314198eeecce4. Report an issue: GitHub.