Mintplex-Labs/anything-llm · error

KoboldCPP must have a valid base path to use for the api.

Error message

KoboldCPP must have a valid base path to use for the api.

What it means

KoboldCPPLLM's constructor throws when KOBOLD_CPP_BASE_PATH is unset, because unlike cloud providers KoboldCPP is a user-hosted OpenAI-compatible endpoint with no default URL. The value becomes the OpenAI SDK baseURL (apiKey is explicitly null), so any chat/embedding operation needs it. AnythingLLM validates this at provider construction rather than at first HTTP call.

Solutions

1. Start KoboldCPP and confirm its OpenAI-compatible endpoint responds (curl http://127.0.0.1:5001/v1/models).
2. Set KOBOLD_CPP_BASE_PATH to that base URL, e.g. KOBOLD_CPP_BASE_PATH=http://127.0.0.1:5001/v1, in the backend .env.
3. Docker users: use http://host.docker.internal:5001/v1 (or the host LAN IP) since localhost inside the container is the container itself.
4. Restart the AnythingLLM server/container so the new env is picked up.
5. Keep KoboldCPP's 'Launch API server' option enabled so /v1 exists.

Example fix

# before (.env) — nothing set

# after (.env)
KOBOLD_CPP_BASE_PATH=http://127.0.0.1:5001/v1
KOBOLD_CPP_MODEL_PREF=<model tag shown in KoboldCPP>

function assertKoboldConfig() {
  const base = process.env.KOBOLD_CPP_BASE_PATH;
  if (!base) throw new Error("KOBOLD_CPP_BASE_PATH missing");
  try { new URL(base); } catch { throw new Error(`KOBOLD_CPP_BASE_PATH not a valid URL: ${base}`); }
}

try {
  llm = new KoboldCPPLLM(embedder, model);
} catch (e) {
  if (/valid base path/i.test(e.message)) return guideUserToEnv("KOBOLD_CPP_BASE_PATH");
  throw e;
}

When it happens

Trigger: Selecting KoboldCPP as the LLM provider while KOBOLD_CPP_BASE_PATH is missing from the server process env. Typically the KoboldCPP local server (e.g., http://127.0.0.1:5001) is not running yet, the env var was never added, or the URL points at the wrong port when the user copied a template for a different tool (e.g., the old KoboldAI horizon API port).

Common situations: Fresh local install where the user launched AnythingLLM before starting KoboldCPP; docker deployments where the host's localhost URL must be host.docker.internal, not localhost; port confusion between the KoboldCPP UI port and its OpenAI-compatible /v1 endpoint; trailing-path mistakes (parseLMStudioBasePath-style normalization is not applied here — the raw URL is used, so include the full base).

Related errors

• No Lemonade API Base Path was set.
• No LMStudio API Base Path was set.
• No LocalAI Base Path was set.
• KoboldCPP must have a valid model set.
• LiteLLM must have a valid base path to use for the api.

AI-assisted analysis of Mintplex-Labs/anything-llm@3aec848f28 (2026-08-18). Data as JSON: /api/errors/d0a829932f0e724d. Report an issue: GitHub.

Appendix: source

Thrown at server/utils/AiProviders/koboldCPP/index.js:16

const { NativeEmbedder } = require("../../EmbeddingEngines/native");
const {
  clientAbortedHandler,
  writeResponseChunk,
  formatChatHistory,
} = require("../../helpers/chat/responses");
const {
  LLMPerformanceMonitor,
} = require("../../helpers/chat/LLMPerformanceMonitor");
const { v4: uuidv4 } = require("uuid");

class KoboldCPPLLM {
  constructor(embedder = null, modelPreference = null) {
    const { OpenAI: OpenAIApi } = require("openai");
    if (!process.env.KOBOLD_CPP_BASE_PATH)
      throw new Error(
        "KoboldCPP must have a valid base path to use for the api."
      );

    this.className = "KoboldCPPLLM";
    this.basePath = process.env.KOBOLD_CPP_BASE_PATH;
    this.openai = new OpenAIApi({
      baseURL: this.basePath,
      apiKey: null,
    });
    this.model = modelPreference ?? process.env.KOBOLD_CPP_MODEL_PREF ?? null;
    if (!this.model) throw new Error("KoboldCPP must have a valid model set.");
    this.limits = {
      history: this.promptWindowLimit() * 0.15,
      system: this.promptWindowLimit() * 0.15,
      user: this.promptWindowLimit() * 0.7,
    };

    this.embedder = embedder ?? new NativeEmbedder();

View on GitHub (pinned to 3aec848f28)