Mintplex-Labs/anything-llm · error · Error

GenericOpenAI must have a valid model set.

Error message

GenericOpenAI must have a valid model set.

What it means

GenericOpenAiLLM throws after construction when this.model resolves to null — modelPreference was null/undefined AND GENERIC_OPEN_AI_MODEL_PREF is unset. Unlike the base path (checked first), the model is optional in construction order: limits are computed and only then is the missing model rejected. The generic provider cannot discover a default model, so it hard-requires one.

Solutions

  1. Set GENERIC_OPEN_AI_MODEL_PREF to a model your endpoint serves (check its /models list) and restart
  2. Or select the model explicitly in the workspace chat settings so modelPreference is non-empty
  3. Verify with curl <base_path>/models that the id you set actually exists on the endpoint

Example fix

# before (.env)
GENERIC_OPEN_AI_BASE_PATH=http://localhost:1234/v1
# model unset -> throws at chat time

# after (.env)
GENERIC_OPEN_AI_BASE_PATH=http://localhost:1234/v1
GENERIC_OPEN_AI_MODEL_PREF=local-model
Defensive patterns

Strategy: validation

Validate before calling

const model = modelPreference ?? process.env.GENERIC_OPEN_AI_MODEL_PREF;
if (!model) {
  return respond("Select a model for the Generic OpenAI endpoint.");
}
const llm = new GenericOpenAiLLM(embedder, model);

Try / catch

try {
  const llm = new GenericOpenAiLLM(embedder, modelPref);
} catch (err) {
  if (/must have a valid model set/.test(err.message)) {
    return respond("No model configured — set GENERIC_OPEN_AI_MODEL_PREF or pick one.");
  }
  throw err;
}

Prevention

When it happens

Trigger: GENERIC_OPEN_AI_BASE_PATH is set but neither the workspace modelPreference nor GENERIC_OPEN_AI_MODEL_PREF is provided. Note modelPreference is read with ?? here, so an explicit empty string from the caller also yields '' which is falsy... in fact '' ?? only skips null/undefined, so '' passes through and still throws via !this.model.

Common situations: Configured the generic endpoint but never picked a model in the workspace; UI submitted the model field empty; env var misspelled (GENERIC_OPENAI_MODEL_PREF vs GENERIC_OPEN_AI_MODEL_PREF); server restarted with a trimmed .env.

Related errors

AI-assisted analysis of Mintplex-Labs/anything-llm@3aec848f28 (2026-08-18). Data as JSON: /api/errors/8ab6312ac794389b. Report an issue: GitHub.

Appendix: source

Thrown at server/utils/AiProviders/genericOpenAi/index.js:41

      );

    this.className = "GenericOpenAiLLM";
    this.basePath = process.env.GENERIC_OPEN_AI_BASE_PATH;
    this.openai = new OpenAIApi({
      baseURL: this.basePath,
      apiKey: process.env.GENERIC_OPEN_AI_API_KEY ?? null,
      defaultHeaders: {
        "User-Agent": getAnythingLLMUserAgent(),
        ...GenericOpenAiLLM.parseCustomHeaders(),
      },
    });
    this.model =
      modelPreference ?? process.env.GENERIC_OPEN_AI_MODEL_PREF ?? null;
    this.maxTokens = process.env.GENERIC_OPEN_AI_MAX_TOKENS
      ? toValidNumber(process.env.GENERIC_OPEN_AI_MAX_TOKENS, 1024)
      : 1024;
    if (!this.model)
      throw new Error("GenericOpenAI must have a valid model set.");
    this.limits = {
      history: this.promptWindowLimit() * 0.15,
      system: this.promptWindowLimit() * 0.15,
      user: this.promptWindowLimit() * 0.7,
    };

    this.embedder = embedder ?? new NativeEmbedder();
    this.defaultTemp = 0.7;
    this.log(`Inference API: ${this.basePath} Model: ${this.model}`);
  }

  log(text, ...args) {
    console.log(`\x1b[36m[${this.className}]\x1b[0m ${text}`, ...args);
  }

  /**
   * Parses custom headers from a CSV-formatted environment variable.
   * Format: "Header-Name:value,Another-Header:value2"

View on GitHub (pinned to 3aec848f28)