Mintplex-Labs/anything-llm · error · Error

${e.message}

Error message

${e.message}

What it means

Re-thrown from the OpenAI SDK rejection inside TextGenWebUILLM.getChatCompletion via `.catch((e) => { throw new Error(e.message); })`. Because TextGenWebUI isValidChatCompletionModel always returns true and this.model is null (the constructor hardcodes it), there is no pre-flight model check — so any server-side rejection (unknown model, auth, connection) surfaces here with only the message string.

Source

Thrown at server/utils/AiProviders/textGenWebUI/index.js:135

      prompt,
      ...formatChatHistory(chatHistory, this.#generateContent),
      {
        role: "user",
        content: this.#generateContent({ userPrompt, attachments }),
      },
    ];
  }

  async getChatCompletion(messages = null, { temperature = 0.7 }) {
    const result = await LLMPerformanceMonitor.measureAsyncFunction(
      this.openai.chat.completions
        .create({
          model: this.model,
          messages,
          temperature,
        })
        .catch((e) => {
          throw new Error(e.message);
        })
    );

    if (
      !result.output.hasOwnProperty("choices") ||
      result.output.choices.length === 0
    )
      return null;

    return {
      textResponse: result.output.choices[0].message.content,
      metrics: {
        prompt_tokens: result.output.usage?.prompt_tokens || 0,
        completion_tokens: result.output.usage?.completion_tokens || 0,
        total_tokens: result.output.usage?.total_tokens || 0,
        outputTps: result.output.usage?.completion_tokens / result.duration,
        duration: result.duration,
        model: this.model,

View on GitHub (pinned to 526360e320)

Solutions

1. Confirm a model is loaded in text-generation-webui and GET <base>/models returns it.
2. Verify the base path/port and that the openai extension is enabled: `curl -s http://127.0.0.1:5001/v1/models`.
3. If the extension requires auth, set TEXT_GEN_WEB_UI_API_KEY to match.
4. For transient connection resets, add bounded retry; for 'model not found', load the model in the webui first.

Example fix

// before
const out = await llm.getChatCompletion(messages, { temperature: 0.7 });

// after
try {
  const out = await llm.getChatCompletion(messages, { temperature: 0.7 });
} catch (err) {
  if (/ECONNREFUSED|fetch failed/i.test(err.message)) throw new Error("TextGenWebUI unreachable — is the openai extension running?", { cause: err });
  if (/model/i.test(err.message)) throw new Error("No model loaded in text-generation-webui", { cause: err });
  throw err;
}

async function assertTextGenReady(basePath, apiKey) {
  // No client-side model check exists (this.model is null), so probe the server
  const res = await fetch(`${basePath}/models`, {
    headers: apiKey ? { Authorization: `Bearer ${apiKey}` } : {},
  });
  if (!res.ok) throw new Error(`text-generation-webui openai extension not ready (${res.status}) at ${basePath}`);
  const { data } = await res.json();
  if (!data || !data.length) throw new Error("No model loaded in text-generation-webui");
}
await assertTextGenReady(process.env.TEXT_GEN_WEB_UI_BASE_PATH, process.env.TEXT_GEN_WEB_UI_API_KEY);

function isTextGenReachableError(message) {
  return typeof message === "string" && /ECONNREFUSED|fetch failed|socket hang up|timeout/i.test(message);
}

try {
  return await llm.getChatCompletion(messages, opts);
} catch (e) {
  if (/ECONNREFUSED|fetch failed/i.test(e.message)) throw new Error("TextGenWebUI unreachable — start the openai extension", { cause: e });
  if (/model/i.test(e.message)) throw new Error("No model loaded in text-generation-webui — load one first", { cause: e });
  if (/401|unauthorized/i.test(e.message)) throw new Error("TextGenWebUI api key mismatch", { cause: e });
  throw e;
}

When it happens

Trigger: The text-generation-webui openai extension returns non-2xx or is unreachable during the create call: connection refused, 404 (no model loaded / wrong model name in the payload), 401 (api key required by the extension but TEXT_GEN_WEB_UI_API_KEY unset/wrong), 500 (model not loaded in the webui), or transport/abort errors.

Common situations: No model loaded in text-generation-webui when the request arrives; the extension is on a different port than TEXT_GEN_WEB_UI_BASE_PATH; the extension expects an api key but TEXT_GEN_WEB_UI_API_KEY is unset (the constructor passes null in that case); the webui process was restarted and the model was not reloaded; client aborted.

Related errors

• ${e.message}
• ${e.message}
• ${e.message}
• ${e.message}
• e.message

AI-assisted analysis of Mintplex-Labs/anything-llm@526360e320 (2026-08-13). Data as JSON: /api/errors/319abd50cfe2234a. Report an issue: GitHub.