Mintplex-Labs/anything-llm · error · Error

NVIDIA NIM chat: ${this.model} is not valid or defined model

Error message

NVIDIA NIM chat: ${this.model} is not valid or defined model for chat completion!

What it means

Unlike the other providers' isValidChatCompletionModel catalog check, NvidiaNimLLM.getChatCompletion simply does `if (!this.model)` — a presence check only. isValidChatCompletionModel for NIM is a stub returning true. this.model = modelPreference || process.env.NVIDIA_NIM_LLM_MODEL_PREF with NO default, so the throw fires when neither is provided. The message ('not valid or defined') is accurate: it's really 'not defined'.

Source

Thrown at server/utils/AiProviders/nvidiaNim/index.js:157

    attachments = [],
  }) {
    const prompt = {
      role: "system",
      content: `${systemPrompt}${this.#appendContext(contextTexts)}`,
    };
    return [
      prompt,
      ...formatChatHistory(chatHistory, this.#generateContent),
      {
        role: "user",
        content: this.#generateContent({ userPrompt, attachments }),
      },
    ];
  }

  async getChatCompletion(messages = null, { temperature = 0.7 }) {
    if (!this.model)
      throw new Error(
        `NVIDIA NIM chat: ${this.model} is not valid or defined model for chat completion!`
      );

    const result = await LLMPerformanceMonitor.measureAsyncFunction(
      this.nvidiaNim.chat.completions
        .create({
          model: this.model,
          messages,
          temperature,
        })
        .catch((e) => {
          throw new Error(e.message);
        })
    );

    if (
      !result.output.hasOwnProperty("choices") ||
      result.output.choices.length === 0

View on GitHub (pinned to 526360e320)

Solutions

1. Set NVIDIA_NIM_LLM_MODEL_PREF to a model id the NIM endpoint actually serves.
2. Or pass modelPreference when constructing NvidiaNimLLM.
3. Confirm the model is loaded/available on the NIM endpoint (GET /v1/models).
4. Check for typos in the env var name.

Example fix

// before
// NVIDIA_NIM_LLM_MODEL_PREF unset, no modelPreference passed -> !this.model -> throws

// after
NVIDIA_NIM_LLM_MODEL_PREF=meta/llama-3.1-8b-instruct

// Ensure a model is configured before calling getChatCompletion
if (!process.env.NVIDIA_NIM_LLM_MODEL_PREF && !modelPreference) {
  throw new Error('No NIM model selected — set NVIDIA_NIM_LLM_MODEL_PREF to a model the endpoint serves.');
}
// Optionally confirm the endpoint actually serves it
const served = await fetch(`${basePath}/models`).then(r => r.json());
const ids = served.data?.map(m => m.id) ?? [];
if (!ids.includes(process.env.NVIDIA_NIM_LLM_MODEL_PREF)) {
  throw new Error(`NIM endpoint does not serve '${process.env.NVIDIA_NIM_LLM_MODEL_PREF}'. Available: ${ids.join(', ')}`);
}

function hasNimModel(model) {
  return typeof model === 'string' && model.trim().length > 0;
}

try {
  await llm.getChatCompletion(messages, { temperature });
} catch (e) {
  if (/NVIDIA NIM chat:.*not valid or defined/i.test(e.message)) {
    // model undefined — set NVIDIA_NIM_LLM_MODEL_PREF or pass modelPreference
  }
}

When it happens

Trigger: Calling getChatCompletion when neither the constructor's modelPreference arg nor NVIDIA_NIM_LLM_MODEL_PREF env var was set — this.model is undefined/empty.

Common situations: Self-hosted NIM selected without specifying which served model to target; env var typo; caller (provider factory) not passing modelPreference; NIM endpoint serves multiple models and none was chosen.

Related errors

• No NVIDIA NIM API Base Path was set.
• Minimax chat: ${this.model} is not valid for chat completion
• Minimax stream: ${this.model} is not valid for chat completi
• Mistral chat: ${this.model} is not valid for chat completion
• Novita chat: ${this.model} is not valid for chat completion!

AI-assisted analysis of Mintplex-Labs/anything-llm@526360e320 (2026-08-13). Data as JSON: /api/errors/53fd485b67765b8c. Report an issue: GitHub.