Mintplex-Labs/anything-llm · warning · Error

No NVIDIA NIM token context limit was set.

Error message

No NVIDIA NIM token context limit was set.

What it means

static promptWindowLimit reads NVIDIA_NIM_LLM_MODEL_TOKEN_LIMIT with a `|| 4096` fallback, then throws 'No ... token context limit was set' if the value is falsy or non-numeric. The `|| 4096` already prevents unset/empty from reaching the check, so in practice this throws ONLY when the env var is set to a non-numeric string (e.g. '8k', '4,096', 'unlimited'). The message ('not set') is therefore misleading — it really means 'not a number'.

Source

Thrown at server/utils/AiProviders/nvidiaNim/index.js:89

        return [];
      });

    if (!model.length) return;
    const modelInfo = model.find((model) => model.id === modelId);
    if (!modelInfo) return;
    process.env.NVIDIA_NIM_LLM_MODEL_TOKEN_LIMIT = Number(
      modelInfo.max_model_len || 4096
    );
  }

  streamingEnabled() {
    return "streamGetChatCompletion" in this;
  }

  static promptWindowLimit(_modelName) {
    const limit = process.env.NVIDIA_NIM_LLM_MODEL_TOKEN_LIMIT || 4096;
    if (!limit || isNaN(Number(limit)))
      throw new Error("No NVIDIA NIM token context limit was set.");
    return Number(limit);
  }

  // Ensure the user set a value for the token limit
  // and if undefined - assume 4096 window.
  promptWindowLimit() {
    const limit = process.env.NVIDIA_NIM_LLM_MODEL_TOKEN_LIMIT || 4096;
    if (!limit || isNaN(Number(limit)))
      throw new Error("No NVIDIA NIM token context limit was set.");
    return Number(limit);
  }

  async isValidChatCompletionModel(_ = "") {
    return true;
  }

  /**
   * Generates appropriate content array for a message + attachments.

View on GitHub (pinned to 526360e320)

Solutions

1. Set NVIDIA_NIM_LLM_MODEL_TOKEN_LIMIT to a plain integer string, e.g. 8192.
2. Remove any unit suffixes or thousands separators.
3. If you intended the default, simply unset the variable (it defaults to 4096).
4. Verify with Number(process.env.NVIDIA_NIM_LLM_MODEL_TOKEN_LIMIT) in a REPL.

Example fix

// before
NVIDIA_NIM_LLM_MODEL_TOKEN_LIMIT=8k   // Number('8k') -> NaN -> throws

// after
NVIDIA_NIM_LLM_MODEL_TOKEN_LIMIT=8192

// Validate the token limit is a plain integer before the static call
const raw = process.env.NVIDIA_NIM_LLM_MODEL_TOKEN_LIMIT;
if (raw != null && raw !== '' && !/^\d+$/.test(String(raw).trim())) {
  throw new Error(`NVIDIA_NIM_LLM_MODEL_TOKEN_LIMIT='${raw}' is not a plain integer.`);
}

function isValidTokenLimit(raw) {
  return raw == null || raw === '' || (/^\d+$/.test(String(raw).trim()) && Number(raw) > 0);
}

try {
  NvidiaNimLLM.promptWindowLimit(modelName);
} catch (e) {
  if (/No NVIDIA NIM token context limit/i.test(e.message)) {
    // the env var is set but non-numeric — fix or unset it (defaults to 4096)
  }
}

When it happens

Trigger: Setting NVIDIA_NIM_LLM_MODEL_TOKEN_LIMIT to a value Number() cannot parse to a finite number — e.g. '8k', '4096 tokens', '4,096', or any non-numeric string. Unset/empty values fall back to 4096 and do NOT throw.

Common situations: User adds a unit suffix ('8k') or thousands separator ('4,096'); copies a human-readable string instead of a plain integer; the static path is hit during class-level lookups before an instance is built.

Related errors

• No token context limit was set.
• No token context limit was set.
• No token context limit was set.
• No token context limit was set.
• No LocalAi token context limit was set.

AI-assisted analysis of Mintplex-Labs/anything-llm@526360e320 (2026-08-13). Data as JSON: /api/errors/a8ceeb62b9ee349e. Report an issue: GitHub.