Mintplex-Labs/anything-llm · error

LiteLLM must have a valid model set.

Error message

LiteLLM must have a valid model set.

What it means

LiteLLM class throws when this.model resolves to null: this.model = modelPreference ?? LITE_LLM_MODEL_PREF ?? null. LiteLLM requires an explicit model string because it is a router — the model id (which may be provider-prefixed like 'gpt-4o-mini' or 'ollama/llama3') determines where the proxy forwards the request. Note ?? semantics: an empty-string modelPreference suppresses the env fallback and triggers this throw.

Solutions

1. Set LITE_LLM_MODEL_PREF to a model the proxy can route, using its exact alias (e.g., LITE_LLM_MODEL_PREF=ollama/llama3 or model-name from litellm config.yaml).
2. If constructing directly, pass a non-empty second argument and never pass "".
3. Verify the alias exists: curl $LITE_LLM_BASE_PATH/v1/models with the API key.
4. Restart the server after env edits.

Example fix

# before (.env)
LITE_LLM_BASE_PATH=http://127.0.0.1:4000

# after (.env)
LITE_LLM_BASE_PATH=http://127.0.0.1:4000
LITE_LLM_MODEL_PREF=gpt-4o-mini

// normalize "" to null so ?? fallback behaves as intended
const liteModel = modelPreference?.trim() || process.env.LITE_LLM_MODEL_PREF || null;
if (!liteModel) throw new Error("LITE_LLM_MODEL_PREF or modelPreference required");
const llm = new LiteLLM(embedder, liteModel);

function hasLiteModel(arg) {
  const m = arg ?? process.env.LITE_LLM_MODEL_PREF ?? null;
  return typeof m === "string" && m.trim().length > 0;
}

try {
  new LiteLLM(embedder, pref);
} catch (e) {
  if (/valid model set/i.test(e.message)) throw new Error("Set LITE_LLM_MODEL_PREF to a model alias the proxy can route");
  throw e;
}

When it happens

Trigger: Constructing the class with modelPreference null/undefined (or the literal "", due to nullish coalescing) while LITE_LLM_MODEL_PREF is unset — i.e., configuring the base path and API key but never naming a model. The constructor computes this.limits from promptWindowLimit() immediately after, so the provider cannot be built modelless.

Common situations: The model var commented out in .env templates; workspace-level model field stored as empty string and passed verbatim; using an old variable name (e.g., LITELLM_MODEL_PREF without the underscore) after an upgrade; expecting the proxy's default model to apply — this client does not have one.

Related errors

• KoboldCPP must have a valid model set.
• LiteLLM must have a valid base path to use for the api.
• LMStudio must have a valid model set.
• No Lemonade Model Pref was set.
• No token context limit was set.

AI-assisted analysis of Mintplex-Labs/anything-llm@3aec848f28 (2026-08-18). Data as JSON: /api/errors/7517ff8bb6234abf. Report an issue: GitHub.

Appendix: source

Thrown at server/utils/AiProviders/liteLLM/index.js:26

} = require("../../helpers/chat/responses");

class LiteLLM {
  constructor(embedder = null, modelPreference = null) {
    const { OpenAI: OpenAIApi } = require("openai");
    if (!process.env.LITE_LLM_BASE_PATH)
      throw new Error(
        "LiteLLM must have a valid base path to use for the api."
      );

    this.className = "LiteLLM";
    this.basePath = process.env.LITE_LLM_BASE_PATH;
    this.openai = new OpenAIApi({
      baseURL: this.basePath,
      apiKey: process.env.LITE_LLM_API_KEY ?? null,
    });
    this.model = modelPreference ?? process.env.LITE_LLM_MODEL_PREF ?? null;

    if (!this.model) throw new Error("LiteLLM must have a valid model set.");
    this.limits = {
      history: this.promptWindowLimit() * 0.15,
      system: this.promptWindowLimit() * 0.15,
      user: this.promptWindowLimit() * 0.7,
    };

    this.embedder = embedder ?? new NativeEmbedder();
    this.defaultTemp = 0.7;
    this.log(`Inference API: ${this.basePath} Model: ${this.model}`);
  }

  log(text, ...args) {
    console.log(`\x1b[36m[${this.className}]\x1b[0m ${text}`, ...args);
  }

  #appendContext(contextTexts = []) {
    if (!contextTexts || !contextTexts.length) return "";
    return (

View on GitHub (pinned to 3aec848f28)