Mintplex-Labs/anything-llm · error · Error

NVIDIA NIM chat: is not valid or defined model for chat…

Error message

NVIDIA NIM chat: ${this.model} is not valid or defined model for chat completion!

What it means

Unlike most providers, NvidiaNimLLM has no hardcoded default model: this.model = modelPreference || process.env.NVIDIA_NIM_LLM_MODEL_PREF. getChatCompletion checks `if (!this.model)` and throws this error before making any request when both the workspace-selected model and the env default are absent.

Solutions

Pick a model in the workspace chat settings (this supplies modelPreference at construction)
Or set NVIDIA_NIM_LLM_MODEL_PREF in .env to an id the NIM server actually hosts (check: curl $NVIDIA_NIM_LLM_BASE_PATH/models)
Restart the server after setting the env default
Verify the stored workspace chat model is a non-empty string

Example fix

# before
# (no model selected; NVIDIA_NIM_LLM_MODEL_PREF unset)
# after
NVIDIA_NIM_LLM_MODEL_PREF=meta/llama-3.1-8b-instruct

Defensive patterns

Strategy: validation

Validate before calling

const model = modelPreference || process.env.NVIDIA_NIM_LLM_MODEL_PREF;
if (!model)
  throw new Error('No NVIDIA NIM model selected — choose one in workspace settings or set NVIDIA_NIM_LLM_MODEL_PREF');
const llm = new NvidiaNimLLM(embedder, model);

Try / catch

try {
  await llm.getChatCompletion(messages);
} catch (e) {
  if (/NVIDIA NIM chat: .* is not valid or defined/.test(e.message)) {
    // config error: select a model or set NVIDIA_NIM_LLM_MODEL_PREF, then retry
  }
  throw e;
}

Prevention

Always select a model in workspace chat settings when using NIM (there is no built-in default)
Set NVIDIA_NIM_LLM_MODEL_PREF as a deployment-wide fallback
Verify the saved workspace model is a non-empty string before instantiating the provider
Cross-check the id against GET $NVIDIA_NIM_LLM_BASE_PATH/models

When it happens

Trigger: NVIDIA NIM selected as the provider but no model chosen in the workspace chat settings and NVIDIA_NIM_LLM_MODEL_PREF not set; the selected model string is empty (cleared in UI or saved as ''); calling getChatCompletion programmatically without passing modelPreference.

Common situations: Fresh NIM setup where the model dropdown was skipped; .env missing the model preference; UI state cleared after reconfiguring the provider; programmatic use that assumes a default model exists.

Related errors

AI-assisted analysis of Mintplex-Labs/anything-llm@3aec848f28 (2026-08-18). Data as JSON: /api/errors/53fd485b67765b8c. Report an issue: GitHub.

Appendix: source

Thrown at server/utils/AiProviders/nvidiaNim/index.js:157

    attachments = [],
  }) {
    const prompt = {
      role: "system",
      content: `${systemPrompt}${this.#appendContext(contextTexts)}`,
    };
    return [
      prompt,
      ...formatChatHistory(chatHistory, this.#generateContent),
      {
        role: "user",
        content: this.#generateContent({ userPrompt, attachments }),
      },
    ];
  }

  async getChatCompletion(messages = null, { temperature = 0.7 }) {
    if (!this.model)
      throw new Error(
        `NVIDIA NIM chat: ${this.model} is not valid or defined model for chat completion!`
      );

    const result = await LLMPerformanceMonitor.measureAsyncFunction(
      this.nvidiaNim.chat.completions
        .create({
          model: this.model,
          messages,
          temperature,
        })
        .catch((e) => {
          throw new Error(e.message);
        })
    );

    if (
      !result.output.hasOwnProperty("choices") ||
      result.output.choices.length === 0

View on GitHub (pinned to 3aec848f28)