Mintplex-Labs/anything-llm · error · RetryError

error.message

Error message

error.message

What it means

Docker Model Runner provider's `stream()` classifies OpenAI SDK errors. It logs `console.error(error.message, error)` first, then re-throws AuthenticationError, wraps RateLimitError/InternalServerError/APIError as `RetryError(error.message)`, and re-throws the rest. Docker Model Runner is the local Docker Desktop LLM endpoint (typically http://localhost:12434); failures are usually local — model not pulled/loaded, Docker Desktop not running, or the local OAI-compatible socket unreachable.

Source

Thrown at server/utils/agents/aibitat/providers/dockerModelRunner.js:126

    try {
      return await tooledStream(
        this.client,
        this.model,
        messages,
        functions,
        eventHandler,
        { provider: this }
      );
    } catch (error) {
      console.error(error.message, error);
      if (error instanceof OpenAI.AuthenticationError) throw error;
      if (
        error instanceof OpenAI.RateLimitError ||
        error instanceof OpenAI.InternalServerError ||
        error instanceof OpenAI.APIError
      ) {
        throw new RetryError(error.message);
      }
      throw error;
    }
  }

  /**
   * Create a non-streaming completion with tool calling support.
   * Uses native tool calling when supported, otherwise falls back to UnTooled.
   */
  async complete(messages, functions = []) {
    const useNative = await this.supportsNativeToolCalling();

    if (!useNative) {
      return await UnTooled.prototype.complete.call(
        this,
        messages,
        functions,
        this.#handleFunctionCallChat.bind(this)

View on GitHub (pinned to 526360e320)

Solutions

1. Inspect the server console — the full error object is logged before rethrow.
2. Treat RetryError as transient (model load is lazy; a retry often succeeds once loaded).
3. Run `docker model list` / `docker model run` to confirm the model is present and loaded.
4. Confirm Docker Desktop is running with the Model Runner feature enabled and that DOCKER_MODEL_RUNNER_BASE_PATH points at the right socket (default http://localhost:12434).
5. Reduce concurrency against the single local model to avoid RateLimitError.

// Confirm Docker Model Runner is up and the model is loaded before the first stream.
async function assertDockerModelRunnerReady(model) {
  const base = process.env.DOCKER_MODEL_RUNNER_BASE_PATH || 'http://localhost:12434';
  const res = await fetch(`${base}/v1/models`);
  if (!res.ok) throw new Error('Docker Model Runner not reachable at ' + base);
}

function isRetryableProviderError(e) {
  return e instanceof OpenAI.RateLimitError || e instanceof OpenAI.InternalServerError || e instanceof OpenAI.APIError;
}

// Lazy model load often succeeds on retry — retry once with backoff.
try { return await provider.stream(messages, functions, handler); }
catch (e) {
  if (e instanceof OpenAI.AuthenticationError) throw e;
  if (e instanceof RetryError) return await backoffAndRetry();
  throw e;
}

When it happens

Trigger: Local model not loaded (APIError / InternalServerError while it lazy-loads); Docker Desktop or the Model Runner feature not running; DOCKER_MODEL_RUNNER_BASE_PATH wrong; concurrent local requests exceed the single-model queue (RateLimitError).

Common situations: First request after Docker restart triggers model load and times out; requested model slug not pulled (`docker model list` shows nothing); DOCKER_MODEL_RUNNER_BASE_PATH misconfigured; resource constraints on the local box.

Related errors

• error.message
• error.message
• error.message
• error.message
• error.message

AI-assisted analysis of Mintplex-Labs/anything-llm@526360e320 (2026-08-13). Data as JSON: /api/errors/cc0186143582acb9. Report an issue: GitHub.