{"record":{"id":"1d6f11b745bf6589","repo":"decolua/9router","slug":"providerid-errormsg-1d6f11","errorCode":null,"errorMessage":"[${providerId}] ${errorMsg}","messagePattern":"\\[\\$\\{providerId\\}\\] \\$\\{errorMsg\\}","errorType":"http","errorClass":null,"httpStatus":503,"severity":"warning","filePath":"src/sse/handlers/search.js","lineNumber":183,"sourceCode":"    // providerId once we fall back, and error locks must be attributed to it.\n    let credentialProviderId = providerId;\n    let credentials = await getProviderCredentials(providerId, excludeConnectionIds, searchLockKey);\n\n    // Fall back to the related chat provider's credentials when this search\n    // provider has none of its own (one key, chat + search).\n    if (!credentials && fallbackProviderId) {\n      credentials = await getProviderCredentials(fallbackProviderId, excludeConnectionIds, searchLockKey);\n      if (credentials) {\n        credentialProviderId = fallbackProviderId;\n        log.info(\"AUTH\", `\\x1b[32m${providerId} reusing ${fallbackProviderId} credentials\\x1b[0m`);\n      }\n    }\n\n    if (!credentials || credentials.allRateLimited) {\n      if (credentials?.allRateLimited) {\n        const errorMsg = lastError || credentials.lastError || \"Unavailable\";\n        const status = lastStatus || Number(credentials.lastErrorCode) || HTTP_STATUS.SERVICE_UNAVAILABLE;\n        log.warn(\"SEARCH\", `[${providerId}] ${errorMsg} (${credentials.retryAfterHuman})`);\n        return unavailableResponse(status, `[${providerId}] ${errorMsg}`, credentials.retryAfter, credentials.retryAfterHuman);\n      }\n      if (excludeConnectionIds.size === 0) {\n        log.error(\"AUTH\", `No credentials for provider: ${providerId}`);\n        return errorResponse(HTTP_STATUS.BAD_REQUEST, `No credentials for provider: ${providerId}`);\n      }\n      log.warn(\"SEARCH\", \"No more accounts available\", { provider: providerId });\n      return errorResponse(lastStatus || HTTP_STATUS.SERVICE_UNAVAILABLE, lastError || \"All accounts unavailable\");\n    }\n\n    log.info(\"AUTH\", `\\x1b[32mUsing ${providerId} account: ${credentials.connectionName}\\x1b[0m`);\n\n    const refreshedCredentials = await checkAndRefreshToken(providerId, credentials);\n\n    const result = await handleSearchCore({\n      body: coreBody,\n      provider: resolvedProvider,\n      providerConfig,","sourceCodeStart":165,"sourceCodeEnd":201,"githubUrl":"https://github.com/decolua/9router/blob/90b52e06ffd666b7929554211474d01588f6b1f8/src/sse/handlers/search.js#L165-L201","documentation":"During the credential fallback loop, getProviderCredentials reports `allRateLimited`: every account/connection for the provider (including any credentialFallback linked provider) is under a rate-limit/unavailability lock. The handler returns unavailableResponse with the stored status, the last upstream error, and a Retry-After hint. The message is '[providerId] <errorMsg>' with the retry window logged.","triggerScenarios":"All configured accounts for the search provider were marked unavailable by markAccountUnavailable after upstream 429/5xx errors; the shared `websearch:{providerId}` lock is active; retrying before the retryAfter window expires; the linked chat provider's shared key was rate-limited, taking the search provider with it.","commonSituations":"Bursty search traffic exhausting per-key quotas; free-tier API keys with tiny rate limits; one bad upstream incident poisoning all accounts; multiple dev machines sharing a single API key; expired OAuth credentials repeatedly failing and refreshing locks.","solutions":["Wait until the Retry-After / retryAfterHuman timestamp before retrying.","Add more credentials/connections for the provider in the dashboard so the fallback pool is non-empty.","Inspect the dashboard account error state and clear the lock (clearAccountError) if the upstream issue was transient.","Check upstream quota/billing: repeated 429s on all accounts usually means plan limits, not config.","Reduce search request rate or add client-side backoff/jitter."],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"const res = await search(body);\nif (res.status === 429 || res.status === 503) {\n  const retryAfter = Number(res.headers.get('retry-after')) || 60;\n  await new Promise(r => setTimeout(r, retryAfter * 1000));\n  return search(body);\n}","preventionTips":["Honor Retry-After instead of hammering retries.","Provision multiple accounts/keys per provider so the fallback pool is non-empty.","Monitor account lock state in the dashboard and clear stale locks.","Add jittered exponential backoff for search workloads."],"tags":["rate-limit","credentials","web-search","http-429","retry-after"],"backgroundTag":"rate-limit-exceeded","analyzedSha":"90b52e06ffd666b7929554211474d01588f6b1f8","analyzedAt":"2026-08-30T21:05:45.952Z","schemaVersion":2},"datasetVersion":"2026-08-30T23:17:21.991Z"}