Mintplex-Labs/anything-llm · error · Error

e.message

Error message

e.message

What it means

Not a distinct error condition but a catch-and-rethrow wrapper around the OpenAI SDK call inside KoboldCPP's getChatCompletion. The `.catch((e) => { throw new Error(e.message); })` strips the original SDK error type, status code, headers, and cause, leaving only the human-readable message string. Any network, auth, rate-limit, or model-not-found failure from the KoboldCPP OpenAI-compatible endpoint surfaces through this wrapper.

Source

Thrown at server/utils/AiProviders/koboldCPP/index.js:140

      ...formatChatHistory(chatHistory, this.#generateContent),
      {
        role: "user",
        content: this.#generateContent({ userPrompt, attachments }),
      },
    ];
  }

  async getChatCompletion(messages = null, { temperature = 0.7 }) {
    const result = await LLMPerformanceMonitor.measureAsyncFunction(
      this.openai.chat.completions
        .create({
          model: this.model,
          messages,
          temperature,
          max_tokens: this.maxTokens,
        })
        .catch((e) => {
          throw new Error(e.message);
        })
    );

    if (
      !result.output.hasOwnProperty("choices") ||
      result.output.choices.length === 0
    )
      return null;

    const promptTokens = LLMPerformanceMonitor.countTokens(messages);
    const completionTokens = LLMPerformanceMonitor.countTokens([
      { content: result.output.choices[0].message.content },
    ]);

    return {
      textResponse: result.output.choices[0].message.content,
      metrics: {
        prompt_tokens: promptTokens,

View on GitHub (pinned to 526360e320)

Solutions

1. Check that the KoboldCPP server is running and reachable at KOBOLD_CPP_BASE_PATH using curl or a browser.
2. Inspect the raw error message for status codes or model-not-found text — the wrapper preserves the message but discards structured data.
3. Verify the model named in KOBOLD_CPP_MODEL_PREF is currently loaded in the KoboldCPP instance.
4. If the wrapper itself is the problem (you need status codes), refactor the catch to re-throw the original error instead of new Error(e.message).

Example fix

// before — original error type and status are lost
.catch((e) => {
  throw new Error(e.message);
})

// after — preserve the original error for upstream handling
.catch((e) => {
  throw e;
})

async function checkKoboldCPPHealth(basePath) {
  const res = await fetch(`${basePath}/models`);
  if (!res.ok) throw new Error(`KoboldCPP at ${basePath} returned ${res.status}`);
  return true;
}

// Optional pre-flight before calling getChatCompletion:
await checkKoboldCPPHealth(process.env.KOBOLD_CPP_BASE_PATH);

try {
  const result = await koboldcppProvider.getChatCompletion(messages, { temperature: 0.7 });
} catch (e) {
  // e.message contains the original SDK message but type/status are lost.
  // Check for common patterns:
  if (e.message.includes('ECONNREFUSED') || e.message.includes('fetch failed')) {
    console.error('KoboldCPP server is not reachable at', process.env.KOBOLD_CPP_BASE_PATH);
  } else if (e.message.includes('model') && e.message.includes('not')) {
    console.error('Model not loaded on KoboldCPP server:', process.env.KOBOLD_CPP_MODEL_PREF);
  } else {
    console.error('KoboldCPP chat completion failed:', e.message);
  }
}

When it happens

Trigger: Calling `koboldcppProvider.getChatCompletion(messages, { temperature })` when the underlying KoboldCPP server is unreachable, returns a non-200, rejects the API key, reports the model as not loaded, or times out. The OpenAI SDK's `chat.completions.create()` promise rejects, and the catch re-wraps the message.

Common situations: KoboldCPP server was stopped or crashed after AnythingLLM started. The base path URL is wrong or missing the /v1 suffix. The model was unloaded from KoboldCPP between configuration and chat. Rate limiting or GPU OOM on the KoboldCPP host produces a 5xx that the SDK surfaces as an error message.

Related errors

• e.message
• ${e.message}
• ${e.message}
• ${e.message}
• ${e.message}

AI-assisted analysis of Mintplex-Labs/anything-llm@526360e320 (2026-08-13). Data as JSON: /api/errors/e579d830413cf882. Report an issue: GitHub.