decolua/9router · warning
[${providerId}] ${errorMsg}
Error message
[${providerId}] ${errorMsg} What it means
During the credential fallback loop, getProviderCredentials reports `allRateLimited`: every account/connection for the provider (including any credentialFallback linked provider) is under a rate-limit/unavailability lock. The handler returns unavailableResponse with the stored status, the last upstream error, and a Retry-After hint. The message is '[providerId] <errorMsg>' with the retry window logged.
Source
Thrown at src/sse/handlers/search.js:183
// providerId once we fall back, and error locks must be attributed to it.
let credentialProviderId = providerId;
let credentials = await getProviderCredentials(providerId, excludeConnectionIds, searchLockKey);
// Fall back to the related chat provider's credentials when this search
// provider has none of its own (one key, chat + search).
if (!credentials && fallbackProviderId) {
credentials = await getProviderCredentials(fallbackProviderId, excludeConnectionIds, searchLockKey);
if (credentials) {
credentialProviderId = fallbackProviderId;
log.info("AUTH", `\x1b[32m${providerId} reusing ${fallbackProviderId} credentials\x1b[0m`);
}
}
if (!credentials || credentials.allRateLimited) {
if (credentials?.allRateLimited) {
const errorMsg = lastError || credentials.lastError || "Unavailable";
const status = lastStatus || Number(credentials.lastErrorCode) || HTTP_STATUS.SERVICE_UNAVAILABLE;
log.warn("SEARCH", `[${providerId}] ${errorMsg} (${credentials.retryAfterHuman})`);
return unavailableResponse(status, `[${providerId}] ${errorMsg}`, credentials.retryAfter, credentials.retryAfterHuman);
}
if (excludeConnectionIds.size === 0) {
log.error("AUTH", `No credentials for provider: ${providerId}`);
return errorResponse(HTTP_STATUS.BAD_REQUEST, `No credentials for provider: ${providerId}`);
}
log.warn("SEARCH", "No more accounts available", { provider: providerId });
return errorResponse(lastStatus || HTTP_STATUS.SERVICE_UNAVAILABLE, lastError || "All accounts unavailable");
}
log.info("AUTH", `\x1b[32mUsing ${providerId} account: ${credentials.connectionName}\x1b[0m`);
const refreshedCredentials = await checkAndRefreshToken(providerId, credentials);
const result = await handleSearchCore({
body: coreBody,
provider: resolvedProvider,
providerConfig,View on GitHub (pinned to 90b52e06ff)
Solutions
- Wait until the Retry-After / retryAfterHuman timestamp before retrying.
- Add more credentials/connections for the provider in the dashboard so the fallback pool is non-empty.
- Inspect the dashboard account error state and clear the lock (clearAccountError) if the upstream issue was transient.
- Check upstream quota/billing: repeated 429s on all accounts usually means plan limits, not config.
- Reduce search request rate or add client-side backoff/jitter.
Defensive patterns
Strategy: retry
Try / catch
const res = await search(body);
if (res.status === 429 || res.status === 503) {
const retryAfter = Number(res.headers.get('retry-after')) || 60;
await new Promise(r => setTimeout(r, retryAfter * 1000));
return search(body);
} Prevention
- Honor Retry-After instead of hammering retries.
- Provision multiple accounts/keys per provider so the fallback pool is non-empty.
- Monitor account lock state in the dashboard and clear stale locks.
- Add jittered exponential backoff for search workloads.
When it happens
Trigger: All configured accounts for the search provider were marked unavailable by markAccountUnavailable after upstream 429/5xx errors; the shared `websearch:{providerId}` lock is active; retrying before the retryAfter window expires; the linked chat provider's shared key was rate-limited, taking the search provider with it.
Common situations: Bursty search traffic exhausting per-key quotas; free-tier API keys with tiny rate limits; one bad upstream incident poisoning all accounts; multiple dev machines sharing a single API key; expired OAuth credentials repeatedly failing and refreshing locks.
Related errors
- "Too many pending authorization requests. Please try again l
- [${providerId}] ${errorMsg}
- All accounts unavailable
- ${msg} (lastError or "All combo models unavailable")
- MiMo bootstrap failed: ${response.status}
AI-assisted analysis of decolua/9router@90b52e06ff (2026-08-30).
Data as JSON: /api/errors/1d6f11b745bf6589.
Report an issue: GitHub.