Mintplex-Labs/anything-llm · error · RetryError

error.message

Error message

error.message

What it means

LiteLLMProvider.stream() wraps failures from the streaming tool-call path (tooledStream, used only when native tool calling is enabled) against a LiteLLM proxy: OpenAI RateLimitError, InternalServerError, or any APIError is rethrown as RetryError with the original message; AuthenticationError is rethrown raw; unknown errors pass through. The raw error is first logged via console.error(error.message, error), so the server log holds the full SDK error object even though the thrown RetryError keeps only the message. RetryError is aibitat's 'safe to retry' signal (error.js:13). Since APIError is the SDK base for all HTTP errors, a bad model name (404) or 400 from the proxy is wrapped identically to transient 429/5xx.

Source

Thrown at server/utils/agents/aibitat/providers/litellm.js:103

    try {
      return await tooledStream(
        this.client,
        this.model,
        messages,
        functions,
        eventHandler,
        { provider: this }
      );
    } catch (error) {
      console.error(error.message, error);
      if (error instanceof OpenAI.AuthenticationError) throw error;
      if (
        error instanceof OpenAI.RateLimitError ||
        error instanceof OpenAI.InternalServerError ||
        error instanceof OpenAI.APIError
      ) {
        throw new RetryError(error.message);
      }
      throw error;
    }
  }

  /**
   * Create a non-streaming completion with tool calling support.
   * Uses native tool calling when enabled via ENV, otherwise falls back to UnTooled.
   */
  async complete(messages, functions = []) {
    const useNative = await this.supportsNativeToolCalling();

    if (!useNative) {
      return await UnTooled.prototype.complete.call(
        this,
        messages,
        functions,
        this.#handleFunctionCallChat.bind(this)

View on GitHub (pinned to 3aec848f28)

Solutions

1. Read the server console log first — this site prints the full error object; the status code in the message tells you 4xx (config) vs 429/5xx (transient)
2. Verify this.model is a valid deployment name on the proxy (litellm `model_list`) and fix LITE_LLM_MODEL_PREF or the agent's model config
3. Confirm LITE_LLM_BASE_PATH points at the proxy root (e.g. http://host:4000) so /v1/chat/completions resolves
4. For 429 with cooldowns, raise the LiteLLM router rpm/tpm limits or wait out the cooldown window, then re-run the chat
5. For 500s, inspect the LiteLLM proxy logs to see which upstream failed and fix that provider's credentials

Example fix

// before: model pref drifts from proxy deployments
// LITE_LLM_MODEL_PREF=gpt-4o  (not in proxy config.yaml -> 400 -> RetryError)

// after
// config.yaml model_list entry: model_name: agent-fast, litellm_params: {model: gpt-4o-mini}
// LITE_LLM_MODEL_PREF=agent-fast

// Pre-flight: model must exist on the proxy and key must work
if (!process.env.LITE_LLM_BASE_PATH) throw new Error("LITE_LLM_BASE_PATH not set");
const r = await fetch(`${process.env.LITE_LLM_BASE_PATH}/v1/models`, {
  headers: { Authorization: `Bearer ${process.env.LITE_LLM_API_KEY ?? ""}` },
});
if (!r.ok) throw new Error(`LiteLLM proxy pre-flight failed (${r.status})`);

const { RetryError } = require("./server/utils/agents/aibitat/error.js");
const isRetryError = (e) => e instanceof RetryError;
const isProxyTransient = (e) => isRetryError(e) && /429|500|502|503/.test(String(e.message));

try {
  return await provider.stream(messages, functions, eventHandler);
} catch (error) {
  if (error instanceof OpenAI.AuthenticationError) throw error; // key wrong — never retry
  if (error instanceof RetryError) {
    if (isProxyTransient(error)) return withBackoff(() => provider.stream(messages, functions, eventHandler));
    throw error; // 400/404 = deployment name wrong
  }
  throw error;
}

When it happens

Trigger: Streaming chat.completions.create against process.env.LITE_LLM_BASE_PATH where the proxy returns 429 (LiteLLM router rate limit/cooldown), 500/502 when the mapped upstream provider errors, 400/404 when this.model (LITE_LLM_MODEL_PREF or config.model) is not a valid LiteLLM deployment name, or 401 when LITE_LLM_API_KEY does not match the proxy's master key (rethrown unwrapped). Only fires on the tooledStream branch; the UnTooled fallback path has its own handling.

Common situations: LITE_LLM_MODEL_PREF names a model alias not configured in the proxy's config.yaml; proxy master key rotated but LITE_LLM_API_KEY stale; upstream OpenAI/Anthropic key expired so LiteLLM returns 500; proxy rate-limit budget exhausted by other clients sharing the proxy.

Related errors

• ${error.message}
• error.message
• error.message
• error.message
• error.message

AI-assisted analysis of Mintplex-Labs/anything-llm@3aec848f28 (2026-08-18). Data as JSON: /api/errors/c96971e461a1165a. Report an issue: GitHub.