{"record":{"id":"73bbd64e01068a41","repo":"decolua/9router","slug":"provider-model-errormsg-73bbd6","errorCode":null,"errorMessage":"[${provider}/${model}] ${errorMsg}","messagePattern":"\\[\\$\\{provider\\}/\\$\\{model\\}\\] \\$\\{errorMsg\\}","errorType":"http","errorClass":null,"httpStatus":503,"severity":"warning","filePath":"src/sse/handlers/embeddings.js","lineNumber":105,"sourceCode":"    log.info(\"ROUTING\", `${modelStr} → ${provider}/${model}`);\n  } else {\n    log.info(\"ROUTING\", `Provider: ${provider}, Model: ${model}`);\n  }\n\n  // Credential + fallback loop (mirrors handleChat)\n  const excludeConnectionIds = new Set();\n  let lastError = null;\n  let lastStatus = null;\n\n  while (true) {\n    const credentials = await getProviderCredentials(provider, excludeConnectionIds, model);\n\n    // All accounts unavailable\n    if (!credentials || credentials.allRateLimited) {\n      if (credentials?.allRateLimited) {\n        const errorMsg = lastError || credentials.lastError || \"Unavailable\";\n        const status = lastStatus || Number(credentials.lastErrorCode) || HTTP_STATUS.SERVICE_UNAVAILABLE;\n        log.warn(\"EMBEDDINGS\", `[${provider}/${model}] ${errorMsg} (${credentials.retryAfterHuman})`);\n        return unavailableResponse(status, `[${provider}/${model}] ${errorMsg}`, credentials.retryAfter, credentials.retryAfterHuman);\n      }\n      if (excludeConnectionIds.size === 0) {\n        log.error(\"AUTH\", `No credentials for provider: ${provider}`);\n        return errorResponse(HTTP_STATUS.BAD_REQUEST, `No credentials for provider: ${provider}`);\n      }\n      log.warn(\"EMBEDDINGS\", \"No more accounts available\", { provider });\n      return errorResponse(lastStatus || HTTP_STATUS.SERVICE_UNAVAILABLE, lastError || \"All accounts unavailable\");\n    }\n\n    log.info(\"AUTH\", `\\x1b[32mUsing ${provider} account: ${credentials.connectionName}\\x1b[0m`);\n\n    const refreshedCredentials = await checkAndRefreshToken(provider, credentials);\n\n    const result = await handleEmbeddingsCore({\n      body: { ...body, model: `${provider}/${model}` },\n      modelInfo: { provider, model },\n      credentials: refreshedCredentials,","sourceCodeStart":87,"sourceCodeEnd":123,"githubUrl":"https://github.com/decolua/9router/blob/90b52e06ffd666b7929554211474d01588f6b1f8/src/sse/handlers/embeddings.js#L87-L123","documentation":"HTTP 429/503-style unavailableResponse returned when every credential account for the target provider is currently rate-limited or unavailable. The message embeds the last upstream error and includes Retry-After info (credentials.retryAfter / retryAfterHuman). It is the rate-limited branch of the credentials loop in handleEmbeddings.","triggerScenarios":"All connections for the provider were marked unavailable by markAccountUnavailable (429s, quota errors, upstream auth failures), so getProviderCredentials returns { allRateLimited: true } on the next loop iteration.","commonSituations":"Burst embedding jobs exhausting per-account quotas; single account provider hitting its rate cap; upstream provider outage causing all accounts to be flagged unavailable; expired tokens repeatedly failing and tripping the unavailable flag.","solutions":["Wait for the retryAfter period indicated in the response/log before retrying","Add more provider accounts/connections in the dashboard to spread load","Route through a combo with fallback providers for embeddings","Clear the account error state in the dashboard if the failure was transient and already recovered","Check upstream provider quotas and reduce request concurrency"],"exampleFix":"// before\nawait embed(model, inputs); // no backoff\n// after\nconst res = await embed(model, inputs);\nif (res.status === 429) { await sleep(res.retryAfterMs ?? 30000); return embed(model, inputs); }","handlingStrategy":"retry","validationCode":"// Pre-check on the client: spread heavy embedding jobs over time\nconst BATCH_DELAY_MS = 1000;\nfor (const batch of batches) {\n  await embed(batch);\n  await new Promise(r => setTimeout(r, BATCH_DELAY_MS));\n}","typeGuard":null,"tryCatchPattern":"const res = await post('/v1/embeddings', payload);\nif (res.status === 429 || res.status === 503) {\n  const retryAfter = Number(res.headers.get('retry-after')) * 1000 || 30000;\n  await sleep(retryAfter);\n  return post('/v1/embeddings', payload); // bounded retries\n}","preventionTips":["Respect the Retry-After header before re-issuing","Add multiple provider accounts to spread quota","Use combos with fallback providers for large jobs","Cap concurrency and batch sizes"],"tags":["rate-limit","quota","fallback-exhausted"],"backgroundTag":"rate-limit-exceeded","analyzedSha":"90b52e06ffd666b7929554211474d01588f6b1f8","analyzedAt":"2026-08-30T21:05:45.952Z","schemaVersion":2},"datasetVersion":"2026-08-30T23:17:21.991Z"}