Mintplex-Labs/anything-llm · error

KoboldCPP must have a valid model set.

Error message

KoboldCPP must have a valid model set.

What it means

KoboldCPPLLM requires an explicit model id: this.model = modelPreference ?? KOBOLD_CPP_MODEL_PREF ?? null, and the constructor throws if that chain yields null. KoboldCPP's OpenAI-compatible API needs a concrete model tag in every request, and — unlike the LMStudio provider — there is no 'fetch from /models' fallback here, so the value must come from the caller or env.

Solutions

1. Set KOBOLD_CPP_MODEL_PREF to the exact model tag KoboldCPP loaded (visible in its UI or via curl http://<base>/v1/models).
2. If constructing programmatically, pass a non-empty modelPreference: new KoboldCPPLLM(embedder, "koboldcpp/default").
3. Avoid empty-string values: because ?? is nullish, treat "" as null in calling code before passing it.
4. Restart the server after editing .env.

Example fix

// before — empty string defeats the ?? fallback
new KoboldCPPLLM(embedder, "");

// after
new KoboldCPPLLM(embedder, "koboldcpp/default");
// or .env: KOBOLD_CPP_MODEL_PREF=koboldcpp/default

// ?? does NOT treat "" as missing — normalize first
const modelPref = modelPreference || process.env.KOBOLD_CPP_MODEL_PREF || null;
if (!modelPref) throw new Error("KoboldCPP model required");
const llm = new KoboldCPPLLM(embedder, modelPref);

function hasKoboldModel(pref, env) {
  const m = pref ?? env ?? null;
  return typeof m === "string" && m.trim().length > 0;
}

try {
  new KoboldCPPLLM(embedder, rawPref);
} catch (e) {
  if (/valid model set/i.test(e.message)) throw new Error("Set KOBOLD_CPP_MODEL_PREF to a tag from /v1/models");
  throw e;
}

When it happens

Trigger: Constructing KoboldCPPLLM with modelPreference null/undefined while KOBOLD_CPP_MODEL_PREF is unset. Note the ?? (nullish) operator: an empty-string modelPreference does NOT fall through to env, so passing "" programmatically (e.g., from a workspace config field left blank) also produces this throw.

Common situations: KOBOLD_CPP_BASE_PATH set but the model pref line forgotten; a workspace-level model selection stored as empty string; copying env templates for LiteLLM/Ollama and missing the Kobold-specific variable name; using a model tag that exists in the UI but not exposing it via the API's model list.

Related errors

• LiteLLM must have a valid model set.
• KoboldCPP must have a valid base path to use for the api.
• LMStudio must have a valid model set.
• No Lemonade Model Pref was set.
• No token context limit was set.

AI-assisted analysis of Mintplex-Labs/anything-llm@f92433b4ea (2026-09-22). Data as JSON: /api/errors/06768bc7643dcdbb. Report an issue: GitHub.

Appendix: source

Thrown at server/utils/AiProviders/koboldCPP/index.js:27

class KoboldCPPLLM {
  static contextWindowSize = null;

  constructor(embedder = null, modelPreference = null) {
    const { OpenAI: OpenAIApi } = require("openai");
    if (!process.env.KOBOLD_CPP_BASE_PATH)
      throw new Error(
        "KoboldCPP must have a valid base path to use for the api."
      );

    this.className = "KoboldCPPLLM";
    this.basePath = process.env.KOBOLD_CPP_BASE_PATH;
    this.openai = new OpenAIApi({
      baseURL: this.basePath,
      apiKey: null,
    });
    this.model = modelPreference ?? process.env.KOBOLD_CPP_MODEL_PREF ?? null;
    if (!this.model) throw new Error("KoboldCPP must have a valid model set.");

    this.embedder = embedder ?? new NativeEmbedder();
    this.defaultTemp = 0.7;
    this.maxTokens = process.env.KOBOLD_CPP_MAX_TOKENS
      ? Number(process.env.KOBOLD_CPP_MAX_TOKENS)
      : null;

    this.limits = null;
    KoboldCPPLLM.cacheContextWindow();
    this.log(
      `Inference API: ${this.basePath} Model: ${this.model} Context Window: ${this.promptWindowLimit()}`
    );
  }

  log(text, ...args) {
    console.log(`\x1b[36m[${this.className}]\x1b[0m ${text}`, ...args);
  }

View on GitHub (pinned to f92433b4ea)