decolua/9router · error
All accounts unavailable
Error message
All accounts unavailable
What it means
HTTP 503 (or lastStatus) returned when the provider has credentials configured but every candidate account was tried and excluded via markAccountUnavailable during this request's fallback loop — none succeeded and none are flagged merely rate-limited. lastError carries the final upstream failure reason.
Source
Thrown at src/sse/handlers/embeddings.js:112
let lastError = null;
let lastStatus = null;
while (true) {
const credentials = await getProviderCredentials(provider, excludeConnectionIds, model);
// All accounts unavailable
if (!credentials || credentials.allRateLimited) {
if (credentials?.allRateLimited) {
const errorMsg = lastError || credentials.lastError || "Unavailable";
const status = lastStatus || Number(credentials.lastErrorCode) || HTTP_STATUS.SERVICE_UNAVAILABLE;
log.warn("EMBEDDINGS", `[${provider}/${model}] ${errorMsg} (${credentials.retryAfterHuman})`);
return unavailableResponse(status, `[${provider}/${model}] ${errorMsg}`, credentials.retryAfter, credentials.retryAfterHuman);
}
if (excludeConnectionIds.size === 0) {
log.error("AUTH", `No credentials for provider: ${provider}`);
return errorResponse(HTTP_STATUS.BAD_REQUEST, `No credentials for provider: ${provider}`);
}
log.warn("EMBEDDINGS", "No more accounts available", { provider });
return errorResponse(lastStatus || HTTP_STATUS.SERVICE_UNAVAILABLE, lastError || "All accounts unavailable");
}
log.info("AUTH", `\x1b[32mUsing ${provider} account: ${credentials.connectionName}\x1b[0m`);
const refreshedCredentials = await checkAndRefreshToken(provider, credentials);
const result = await handleEmbeddingsCore({
body: { ...body, model: `${provider}/${model}` },
modelInfo: { provider, model },
credentials: refreshedCredentials,
log,
onCredentialsRefreshed: async (newCreds) => {
await updateProviderCredentials(credentials.connectionId, {
...newCreds,
existingProviderSpecificData: credentials.providerSpecificData,
testStatus: "active"
});View on GitHub (pinned to 90b52e06ff)
Solutions
- Inspect the dashboard account list and re-authenticate/fix failing connections
- Check the router log for the underlying lastError of each failed account
- Add a healthy fallback provider or combo route
- Wait for the upstream outage to resolve and clear account error states
- Verify tokens are valid via the dashboard's connection test button
Example fix
// before
await embed({ model: 'provider/x', input }); // fails 503
// after
try {
return await embed({ model: 'provider/x', input });
} catch (e) {
return await embed({ model: 'fallback-provider/x', input }); // combo/alternate route
} Defensive patterns
Strategy: fallback
Validate before calling
// Before large jobs, confirm at least one healthy account via a tiny probe
const probe = await post('/v1/embeddings', { model, input: ['ping'] });
if (!probe.ok && probe.status >= 500) throw new Error('Provider accounts unhealthy — check dashboard'); Try / catch
try {
return await post('/v1/embeddings', payload);
} catch (e) {
if (e.status === 503) return post('/v1/embeddings', { ...payload, model: fallbackModel });
throw e;
} Prevention
- Keep a second provider configured as fallback
- Fix/refresh failing connections promptly in the dashboard
- Alert on lastError appearing in router logs
- Clear stale unavailable flags after upstream recovers
When it happens
Trigger: getProviderCredentials returns null while excludeConnectionIds.size > 0, i.e. the request already cycled through all accounts and each failed with shouldFallback=true.
Common situations: All provider accounts have expired/invalid tokens that refresh cannot fix; upstream provider-wide outage; wrong credentials entered for every account; embedding job large enough to exhaust all accounts' quotas mid-run.
Related errors
AI-assisted analysis of decolua/9router@90b52e06ff (2026-08-30).
Data as JSON: /api/errors/4f04ad4ddd182a3f.
Report an issue: GitHub.