zeroclaw-labs/zeroclaw · warning

xAI token refresh is in backoff for {remaining}s due to prev

Error message

xAI token refresh is in backoff for {remaining}s due to previous failures

What it means

xAI refresh backoff guard: get_valid_xai_access_token needs to refresh (token expiring within 90 s, refresh token present) but refresh_backoff_remaining reports an active window from a failed refresh within the last 10 seconds, so the call bails immediately with the remaining seconds embedded. The backoff map is in-process and per profile_id.

Source

Thrown at crates/zeroclaw-providers/src/auth/mod.rs:427

        };

        let refresh_lock = refresh_lock_for_profile(&profile_id);
        let _guard = refresh_lock.lock().await;

        let data = self.store.load().await?;
        let Some(latest_profile) = data.profiles.get(&profile_id) else {
            return Ok(None);
        };
        let Some(latest_tokens) = latest_profile.token_set.as_ref() else {
            anyhow::bail!("xAI auth profile is missing token set: {profile_id}");
        };
        if !latest_tokens.is_expiring_within(Duration::from_secs(OPENAI_REFRESH_SKEW_SECS)) {
            return Ok(Some(latest_tokens.access_token.clone()));
        }

        let refresh_token = latest_tokens.refresh_token.clone().unwrap_or(refresh_token);
        if let Some(remaining) = refresh_backoff_remaining(&profile_id) {
            anyhow::bail!(
                "xAI token refresh is in backoff for {remaining}s due to previous failures"
            );
        }

        let mut refreshed =
            match refresh_xai_access_token_with_retries(&self.client, &refresh_token).await {
                Ok(tokens) => {
                    clear_refresh_backoff(&profile_id);
                    tokens
                }
                Err(err) => {
                    set_refresh_backoff(
                        &profile_id,
                        Duration::from_secs(OPENAI_REFRESH_FAILURE_BACKOFF_SECS),
                    );
                    return Err(err);
                }
            };

View on GitHub (pinned to 88bb9c8533)

Solutions

  1. Wait the embedded remaining seconds and retry the call
  2. Surface the underlying failure with auth refresh --model-provider xai and fix it (re-login if the refresh token is revoked)
  3. Add your own spacing between credential resolution attempts so you rarely meet the guard
Defensive patterns

Strategy: retry

Validate before calling

// Pace retries wider than the 10s in-process backoff window.
tokio::time::sleep(std::time::Duration::from_secs(10)).await;
let token = auth.get_valid_xai_access_token(None).await?;

Try / catch

match auth.get_valid_xai_access_token(override_).await {
    Ok(tok) => tok,
    Err(e) => {
        let msg = e.to_string();
        if msg.contains("refresh is in backoff") {
            let secs: u64 = msg.split("backoff for ").nth(1)
                .and_then(|r| r.split('s').next()).and_then(|s| s.parse().ok()).unwrap_or(1);
            tokio::time::sleep(std::time::Duration::from_secs(secs + 1)).await;
            return auth.get_valid_xai_access_token(override_).await;
        }
        Err(e)
    }
}

Prevention

When it happens

Trigger: A prior refresh_xai_access_token_with_retries failure (network error, revoked xAI refresh token, auth endpoint 5xx) is followed within 10 s by another resolve_credential or refresh_status call on the same profile.

Common situations: Startup storms where many tasks resolve xAI credentials while the provider endpoint is flaking; revoked refresh token making every attempt fail; retry loops without their own delay stacking onto the guard.

Related errors

AI-assisted analysis of zeroclaw-labs/zeroclaw@88bb9c8533 (2026-08-23). Data as JSON: /api/errors/4cfbf00a2f40571e. Report an issue: GitHub.