Mintplex-Labs/anything-llm · error · RetryError

error.message

Error message

error.message

What it means

Foundry provider's `stream()` catch-all for OpenAI errors that are NOT premature-close and NOT AuthenticationError: `RateLimitError`, `InternalServerError`, or any other `APIError` is wrapped as `RetryError(error.message)` — the framework's transient-retry signal. Foundry Local can emit RateLimitError when its local concurrency queue is full and InternalServerError when local inference fails without a premature-close signature.

Source

Thrown at server/utils/agents/aibitat/providers/foundry.js:192

      return await tooledStream(
        this.client,
        this.model,
        messages,
        functions,
        this.#filterToolCallMarkup(eventHandler),
        { provider: this }
      );
    } catch (error) {
      console.error(error.message, error);
      if (error instanceof OpenAI.AuthenticationError) throw error;
      if (this.#isPrematureClose(error))
        throw new Error(FoundryLLM.explainStreamError(error, this.model));
      if (
        error instanceof OpenAI.RateLimitError ||
        error instanceof OpenAI.InternalServerError ||
        error instanceof OpenAI.APIError
      ) {
        throw new RetryError(error.message);
      }
      throw error;
    }
  }

  /**
   * Create a non-streaming completion with tool calling support.
   * Uses native tool calling when supported, otherwise falls back to UnTooled.
   */
  async complete(messages, functions = []) {
    const useNative = await this.supportsNativeToolCalling();

    if (!useNative) {
      return await UnTooled.prototype.complete.call(
        this,
        messages,
        functions,
        this.#handleFunctionCallChat.bind(this)

View on GitHub (pinned to 526360e320)

Solutions

1. Treat RetryError as transient and let the framework retry the turn.
2. Lower concurrency to a single in-flight Foundry request to avoid RateLimitError.
3. Verify FOUNDRY_MODEL_PREF is a model Foundry Local actually serves.
4. Inspect the Foundry Local logs for the underlying 5xx.
5. If persistent, restart Foundry Local and re-pull the model.

// Confirm the model slug is one Foundry Local serves before the turn.
async function assertFoundryModelAvailable(model) {
  const caps = await new FoundryLLM(null, model).getModelCapabilities();
  if (!caps) throw new Error(`Foundry Local does not serve model: ${model}`);
}

function isRetryableProviderError(e) {
  return e instanceof OpenAI.RateLimitError || e instanceof OpenAI.InternalServerError || e instanceof OpenAI.APIError;
}

// Honor RetryError; keep one in-flight request to dodge Foundry RateLimitError.
try { return await provider.stream(messages, functions, handler); }
catch (e) {
  if (e instanceof OpenAI.AuthenticationError) throw e;
  if (e instanceof RetryError) return await backoffAndRetry();
  throw e;
}

When it happens

Trigger: Foundry Local concurrency limit hit (RateLimitError on the local queue); Foundry Local internal process error that did not manifest as a premature close; bad model slug returned as APIError; transient local GPU error surfaced as InternalServerError.

Common situations: Multiple concurrent agent turns against one Foundry model; FOUNDRY_MODEL_PREF referencing a model Foundry Local does not serve; Foundry service partially degraded after a crash that did not produce a premature-close.

Related errors

• error.message
• error.message
• error.message
• error.message
• error.message

AI-assisted analysis of Mintplex-Labs/anything-llm@526360e320 (2026-08-13). Data as JSON: /api/errors/949d236b026c0678. Report an issue: GitHub.