Mintplex-Labs/anything-llm · error · Error

${e.message}

Error message

${e.message}

What it means

Propagates the raw rejection message from the NVIDIA NIM OpenAI-compatible 'chat.completions.create' call. The NIM client is built with apiKey:null, so failures surface as the underlying SDK/HTTP message (model not found, server unreachable, context overflow, malformed base path). The catch discards the original Error's status/cause and re-throws only e.message.

Source

Thrown at server/utils/AiProviders/nvidiaNim/index.js:169

      },
    ];
  }

  async getChatCompletion(messages = null, { temperature = 0.7 }) {
    if (!this.model)
      throw new Error(
        `NVIDIA NIM chat: ${this.model} is not valid or defined model for chat completion!`
      );

    const result = await LLMPerformanceMonitor.measureAsyncFunction(
      this.nvidiaNim.chat.completions
        .create({
          model: this.model,
          messages,
          temperature,
        })
        .catch((e) => {
          throw new Error(e.message);
        })
    );

    if (
      !result.output.hasOwnProperty("choices") ||
      result.output.choices.length === 0
    )
      return null;

    return {
      textResponse: result.output.choices[0].message.content,
      metrics: {
        prompt_tokens: result.output.usage.prompt_tokens || 0,
        completion_tokens: result.output.usage.completion_tokens || 0,
        total_tokens: result.output.usage.total_tokens || 0,
        outputTps: result.output.usage.completion_tokens / result.duration,
        duration: result.duration,
        model: this.model,

View on GitHub (pinned to 526360e320)

Solutions

1. Read the exact e.message text: 'model not found' vs 'fetch failed' vs 'context length exceeded' points to different roots.
2. Confirm the NIM server is up by curling ${NVIDIA_NIM_LLM_BASE_PATH}/v1/models and checking this.model appears in the list.
3. Verify NVIDIA_NIM_LLM_BASE_PATH and NVIDIA_NIM_LLM_MODEL_PREF are set and that the base path origin is reachable.
4. If the message indicates context length, raise NVIDIA_NIM_LLM_MODEL_TOKEN_LIMIT or trim contextTexts/chatHistory.

Example fix

// before
const result = await LLMPerformanceMonitor.measureAsyncFunction(
  this.nvidiaNim.chat.completions.create({ model: this.model, messages, temperature })
    .catch((e) => { throw new Error(e.message); })
);

// after - preserve status/cause for diagnosis
.catch((e) => {
  const err = new Error(e.message);
  err.status = e.status;
  err.cause = e;
  throw err;
})

const assertNimReady = async (basePath, model) => {
  const res = await fetch(`${basePath.replace(/\/$/, '')}/v1/models`);
  if (!res.ok) throw new Error(`NIM endpoint unhealthy: HTTP ${res.status}`);
  const { data = [] } = await res.json();
  if (!data.some((m) => m.id === model)) throw new Error(`NIM model '${model}' not served`);
};
await assertNimReady(process.env.NVIDIA_NIM_LLM_BASE_PATH, modelId);

const hasNimChoices = (o) =>
  !!o && typeof o === 'object' && Array.isArray(o.choices) && o.choices.length > 0;

try {
  const out = await llm.getChatCompletion(messages, { temperature });
} catch (e) {
  logger.error('NIM chat failed', { message: e.message, status: e.status });
  throw e;
}

When it happens

Trigger: Calling NvidiaNimLLM.getChatCompletion when the NIM container is down, the model id in this.model is not served by the endpoint, the prompt exceeds the model's context window, or NVIDIA_NIM_LLM_BASE_PATH resolves to a URL that returns a non-2xx (e.g. wrong port, missing /v1).

Common situations: NIM Docker container not started; NVIDIA_NIM_LLM_MODEL_PREF left blank or typo'd; base path pasted with a trailing slash or wrong port; model not yet pulled/loaded on the NIM server; oversized prompt after context injection.

Related errors

• res.statusText || "Error finding custom models."
• No NVIDIA NIM API Base Path was set.
• No NVIDIA NIM token context limit was set.
• NVIDIA NIM chat: ${this.model} is not valid or defined model
• OMLX:cacheContextWindows - ${res.statusText}

AI-assisted analysis of Mintplex-Labs/anything-llm@526360e320 (2026-08-13). Data as JSON: /api/errors/c793964772eda3df. Report an issue: GitHub.