Mintplex-Labs/anything-llm · critical · Error

KoboldCPP must have a valid model set.

Error message

KoboldCPP must have a valid model set.

What it means

Thrown by the KoboldCPP provider constructor when no model name can be resolved. The constructor tries the workspace-supplied modelPreference first, then falls back to the KOBOLD_CPP_MODEL_PREF environment variable, and finally defaults to null. If all three resolve to a falsy value, the constructor aborts, preventing the OpenAI-compatible client from being used without a model identifier.

Source

Thrown at server/utils/AiProviders/koboldCPP/index.js:27

} = require("../../helpers/chat/LLMPerformanceMonitor");
const { v4: uuidv4 } = require("uuid");

class KoboldCPPLLM {
  constructor(embedder = null, modelPreference = null) {
    const { OpenAI: OpenAIApi } = require("openai");
    if (!process.env.KOBOLD_CPP_BASE_PATH)
      throw new Error(
        "KoboldCPP must have a valid base path to use for the api."
      );

    this.className = "KoboldCPPLLM";
    this.basePath = process.env.KOBOLD_CPP_BASE_PATH;
    this.openai = new OpenAIApi({
      baseURL: this.basePath,
      apiKey: null,
    });
    this.model = modelPreference ?? process.env.KOBOLD_CPP_MODEL_PREF ?? null;
    if (!this.model) throw new Error("KoboldCPP must have a valid model set.");
    this.limits = {
      history: this.promptWindowLimit() * 0.15,
      system: this.promptWindowLimit() * 0.15,
      user: this.promptWindowLimit() * 0.7,
    };

    this.embedder = embedder ?? new NativeEmbedder();
    this.defaultTemp = 0.7;
    this.maxTokens = Number(process.env.KOBOLD_CPP_MAX_TOKENS) || 2048;
    this.log(`Inference API: ${this.basePath} Model: ${this.model}`);
  }

  log(text, ...args) {
    console.log(`\x1b[36m[${this.className}]\x1b[0m ${text}`, ...args);
  }

  #appendContext(contextTexts = []) {
    if (!contextTexts || !contextTexts.length) return "";

View on GitHub (pinned to 526360e320)

Solutions

1. Set KOBOLD_CPP_MODEL_PREF in your .env to a valid model loaded by your KoboldCPP server (e.g. KOBOLD_CPP_MODEL_PREF='koboldcpp/codellama-7b-instruct.Q4_K_S') and restart the server.
2. Select a specific model in the AnythingLLM workspace LLM settings UI so the modelPreference argument is populated.
3. Verify the model name matches what KoboldCPP reports via its /v1/models endpoint — a mismatch won't throw here but will fail later.

Example fix

// before: .env has base path but no model
KOBOLD_CPP_BASE_PATH='http://localhost:5001/v1'

// after: add the model pref
KOBOLD_CPP_BASE_PATH='http://localhost:5001/v1'
KOBOLD_CPP_MODEL_PREF='koboldcpp/codellama-7b-instruct.Q4_K_S'

function validateKoboldCPPModel(modelPreference) {
  const model = modelPreference ?? process.env.KOBOLD_CPP_MODEL_PREF ?? null;
  if (!model) {
    throw new Error(
      'KOBOLD_CPP_MODEL_PREF is not set and no model was provided. ' +
      'Set KOBOLD_CPP_MODEL_PREF in .env or pass a model name.'
    );
  }
  return model;
}

// Call before instantiating the provider:
const model = validateKoboldCPPModel(workspaceModel);
new KoboldCPPLLM(embedder, model);

/** @returns {model is string} */
function isValidModelName(model) {
  return typeof model === 'string' && model.trim().length > 0;
}

let llm;
try {
  llm = new KoboldCPPLLM(embedder, workspaceModel);
} catch (e) {
  if (e.message.includes('valid model set')) {
    // Surface a user-friendly configuration error
    return respondWithConfigError('Please select a KoboldCPP model in settings.');
  }
  throw e;
}

When it happens

Trigger: Instantiating `new KoboldCPPLLM(embedder, model)` (or selecting the 'koboldcpp' provider through getLLMProvider) where both the `model` argument is null/undefined/empty-string AND process.env.KOBOLD_CPP_MODEL_PREF is unset or empty. The `??` operator means empty-string from the param is overridden only if truly nullish (null/undefined), not if it is an empty string.

Common situations: User selects KoboldCPP as the LLM provider in workspace settings but never picks a model from the dropdown. The .env file has the KOBOLD_CPP_BASE_PATH set (so it passes the earlier base-path guard) but omits KOBOLD_CPP_MODEL_PREF. A workspace that was previously configured with a model that was later removed or renamed on the KoboldCPP server.

Related errors

• No Lemonade Model Pref was set.
• LiteLLM must have a valid model set.
• LMStudio must have a valid model set.
• GenericOpenAI must have a valid model set.
• KoboldCPP must have a valid base path to use for the api.

AI-assisted analysis of Mintplex-Labs/anything-llm@526360e320 (2026-08-13). Data as JSON: /api/errors/06768bc7643dcdbb. Report an issue: GitHub.