Mintplex-Labs/anything-llm · error · RetryError

${error.message}

Error message

${error.message}

What it means

LocalAiProvider.stream() catches failures from the streaming tool-call request (tooledStream) to a self-hosted LocalAI instance (LOCAL_AI_BASE_PATH) and rethrows OpenAI RateLimitError, InternalServerError, or any APIError as aibitat's RetryError with the original message; AuthenticationError is rethrown raw; other errors pass through. The full error object is logged first via console.error, so the server console has the raw cause. LocalAI commonly returns 500 during model cold-start (first request after a model is installed) and 404 for unknown model ids — both land in this RetryError wrap.

Source

Thrown at server/utils/agents/aibitat/providers/localai.js:110

    try {
      await LocalAiLLM.cacheContextWindows();
      return await tooledStream(
        this.client,
        this.model,
        messages,
        functions,
        eventHandler,
        { provider: this }
      );
    } catch (error) {
      console.error(error.message, error);
      if (error instanceof OpenAI.AuthenticationError) throw error;
      if (
        error instanceof OpenAI.RateLimitError ||
        error instanceof OpenAI.InternalServerError ||
        error instanceof OpenAI.APIError
      ) {
        throw new RetryError(error.message);
      }
      throw error;
    }
  }

  /**
   * Create a non-streaming completion with tool calling support.
   * Uses native tool calling when supported, otherwise falls back to UnTooled.
   */
  async complete(messages, functions = []) {
    const useNative = await this.supportsNativeToolCalling();

    if (!useNative) {
      return await UnTooled.prototype.complete.call(
        this,
        messages,
        functions,
        this.#handleFunctionCallChat.bind(this)

View on GitHub (pinned to 20f6d3546c)

Solutions

1. Check the console log — this site prints the raw error; a 500 right after model install means LocalAI is still loading, wait for the build to finish and retry
2. Verify this.model matches an installed LocalAI model exactly (GET {LOCAL_AI_BASE_PATH}/models)
3. Confirm LOCAL_AI_BASE_PATH includes the full API root, e.g. http://localhost:8080/v1
4. Align LOCAL_AI_API_KEY with the LocalAI server's auth setting (mismatch on protected routes can surface as 403/500 rather than a clean 401)

Example fix

# before
LOCAL_AI_BASE_PATH=http://localhost:8080     # missing /v1 -> 404 APIError -> RetryError

# after
LOCAL_AI_BASE_PATH=http://localhost:8080/v1
LOCAL_AI_API_KEY=my-localai-key

// Pre-flight the LocalAI instance
const base = process.env.LOCAL_AI_BASE_PATH;
if (!base) throw new Error("LOCAL_AI_BASE_PATH not set");
const res = await fetch(`${base}/models`);
if (!res.ok) throw new Error(`LocalAI unreachable (${res.status}) — check base path includes /v1`);

const { RetryError } = require("./server/utils/agents/aibitat/error.js");
const isRetryError = (e) => e instanceof RetryError;
const isColdStart = (e) => isRetryError(e) && /500/.test(String(e.message)); // first call after model install

try {
  return await provider.stream(messages, functions, eventHandler);
} catch (error) {
  if (error instanceof OpenAI.AuthenticationError) throw error;
  if (error instanceof RetryError) {
    if (isColdStart(error)) { await waitForLocalAiModel(base, model); return provider.stream(messages, functions, eventHandler); }
    throw error;
  }
  throw error;
}

When it happens

Trigger: Streaming POST {LOCAL_AI_BASE_PATH}/chat/completions returning 500 while LocalAI is still building/loading the model backend (first call after install), 404 when this.model is not a model LocalAI knows, 429 when LocalAI's concurrency limit is hit, or connection-level APIError when LOCAL_AI_BASE_PATH is wrong (e.g. missing /v1).

Common situations: Fresh LocalAI install: the very first agent chat arrives before the model finished building and 500s; LOCAL_AI_BASE_PATH set without the /v1 suffix or to a port where nothing listens; LOCAL_AI_API_KEY set but LocalAI started without an API key gate (or vice versa); model name pulled from config doesn't match the LocalAI models folder.

Related errors

• error.message
• ${error.message}
• error.message
• error.message
• error.message

AI-assisted analysis of Mintplex-Labs/anything-llm@20f6d3546c (2026-08-18). Data as JSON: /api/errors/6415c8362714ee1f. Report an issue: GitHub.