{"record":{"id":"9dc2c7840d9c22be","repo":"decolua/9router","slug":"all-accounts-unavailable-9dc2c7","errorCode":null,"errorMessage":"All accounts unavailable","messagePattern":"All accounts unavailable","errorType":"http","errorClass":null,"httpStatus":503,"severity":"error","filePath":"src/sse/handlers/search.js","lineNumber":190,"sourceCode":"      credentials = await getProviderCredentials(fallbackProviderId, excludeConnectionIds, searchLockKey);\n      if (credentials) {\n        credentialProviderId = fallbackProviderId;\n        log.info(\"AUTH\", `\\x1b[32m${providerId} reusing ${fallbackProviderId} credentials\\x1b[0m`);\n      }\n    }\n\n    if (!credentials || credentials.allRateLimited) {\n      if (credentials?.allRateLimited) {\n        const errorMsg = lastError || credentials.lastError || \"Unavailable\";\n        const status = lastStatus || Number(credentials.lastErrorCode) || HTTP_STATUS.SERVICE_UNAVAILABLE;\n        log.warn(\"SEARCH\", `[${providerId}] ${errorMsg} (${credentials.retryAfterHuman})`);\n        return unavailableResponse(status, `[${providerId}] ${errorMsg}`, credentials.retryAfter, credentials.retryAfterHuman);\n      }\n      if (excludeConnectionIds.size === 0) {\n        log.error(\"AUTH\", `No credentials for provider: ${providerId}`);\n        return errorResponse(HTTP_STATUS.BAD_REQUEST, `No credentials for provider: ${providerId}`);\n      }\n      log.warn(\"SEARCH\", \"No more accounts available\", { provider: providerId });\n      return errorResponse(lastStatus || HTTP_STATUS.SERVICE_UNAVAILABLE, lastError || \"All accounts unavailable\");\n    }\n\n    log.info(\"AUTH\", `\\x1b[32mUsing ${providerId} account: ${credentials.connectionName}\\x1b[0m`);\n\n    const refreshedCredentials = await checkAndRefreshToken(providerId, credentials);\n\n    const result = await handleSearchCore({\n      body: coreBody,\n      provider: resolvedProvider,\n      providerConfig,\n      credentials: refreshedCredentials,\n      log,\n      onCredentialsRefreshed: async (newCreds) => {\n        await updateProviderCredentials(credentials.connectionId, {\n          accessToken: newCreds.accessToken,\n          refreshToken: newCreds.refreshToken,\n          providerSpecificData: newCreds.providerSpecificData,","sourceCodeStart":172,"sourceCodeEnd":208,"githubUrl":"https://github.com/decolua/9router/blob/90b52e06ffd666b7929554211474d01588f6b1f8/src/sse/handlers/search.js#L172-L208","documentation":"After at least one account was tried and excluded (excludeConnectionIds non-empty), the loop ran out of accounts: getProviderCredentials returned nothing and not allRateLimited. The handler returns lastStatus (or 503) with lastError, defaulting the message to 'All accounts unavailable'. It means every credential for the provider failed and was excluded during this request.","triggerScenarios":"Multi-account providers where each account hit a fallback-eligible error (5xx, auth failure) in sequence during one request; lastError unset when the first failure carried no message, so the default text appears; concurrent requests exhausting the shared account pool.","commonSituations":"All OAuth tokens expired simultaneously (refresh failing); provider-wide outage returning 502/503 for every account; accounts disabled/invalid in the dashboard but still listed; one shared key used across many clients tripping per-account limits.","solutions":["Read the accompanying `lastError` in the response/log — it names why the first account failed.","Re-authenticate or update credentials for the provider's connections in the dashboard.","Verify upstream status: a provider-wide outage recovers on its own; retry with backoff.","Add healthy accounts to the pool; remove dead connections that burn fallback attempts.","Check token refresh logs (checkAndRefreshToken) for repeated refresh failures."],"exampleFix":null,"handlingStrategy":"fallback","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  return await searchWith(provider, body);\n} catch (e) {\n  if (e.status === 503 && /accounts unavailable/.test(e.message)) {\n    return await searchWith(fallbackProvider, body); // different provider entirely\n  }\n  throw e;\n}","preventionTips":["Configure at least two independent accounts for critical search providers.","Alert on repeated account-unavailable marks per provider.","Keep OAuth refresh tokens valid; re-auth before expiry.","Read lastError in the response — it names the first account's failure cause."],"tags":["credentials","account-fallback","web-search","http-503"],"backgroundTag":"all-accounts-exhausted","analyzedSha":"90b52e06ffd666b7929554211474d01588f6b1f8","analyzedAt":"2026-08-30T21:05:45.952Z","schemaVersion":2},"datasetVersion":"2026-08-30T23:17:21.991Z"}