{"record":{"id":"a8ceeb62b9ee349e","repo":"Mintplex-Labs/anything-llm","slug":"no-nvidia-nim-token-context-limit-was-set","errorCode":null,"errorMessage":"No NVIDIA NIM token context limit was set.","messagePattern":"No NVIDIA NIM token context limit was set\\.","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"server/utils/AiProviders/nvidiaNim/index.js","lineNumber":89,"sourceCode":"        return [];\n      });\n\n    if (!model.length) return;\n    const modelInfo = model.find((model) => model.id === modelId);\n    if (!modelInfo) return;\n    process.env.NVIDIA_NIM_LLM_MODEL_TOKEN_LIMIT = Number(\n      modelInfo.max_model_len || 4096\n    );\n  }\n\n  streamingEnabled() {\n    return \"streamGetChatCompletion\" in this;\n  }\n\n  static promptWindowLimit(_modelName) {\n    const limit = process.env.NVIDIA_NIM_LLM_MODEL_TOKEN_LIMIT || 4096;\n    if (!limit || isNaN(Number(limit)))\n      throw new Error(\"No NVIDIA NIM token context limit was set.\");\n    return Number(limit);\n  }\n\n  // Ensure the user set a value for the token limit\n  // and if undefined - assume 4096 window.\n  promptWindowLimit() {\n    const limit = process.env.NVIDIA_NIM_LLM_MODEL_TOKEN_LIMIT || 4096;\n    if (!limit || isNaN(Number(limit)))\n      throw new Error(\"No NVIDIA NIM token context limit was set.\");\n    return Number(limit);\n  }\n\n  async isValidChatCompletionModel(_ = \"\") {\n    return true;\n  }\n\n  /**\n   * Generates appropriate content array for a message + attachments.","sourceCodeStart":71,"sourceCodeEnd":107,"githubUrl":"https://github.com/Mintplex-Labs/anything-llm/blob/3aec848f2885144aa8f1e53b9731a04310d5d558/server/utils/AiProviders/nvidiaNim/index.js#L71-L107","documentation":"Static variant of NvidiaNimLLM.promptWindowLimit(): it reads NVIDIA_NIM_LLM_MODEL_TOKEN_LIMIT with a 4096 fallback and throws when the value cannot be converted to a number. Since unset/empty falls back to 4096, the throw means the variable is set to a non-numeric string. The static form can be invoked without an instance (e.g. for preflight sizing), before any model sync has normalized the value.","triggerScenarios":"Setting NVIDIA_NIM_LLM_MODEL_TOKEN_LIMIT to a non-integer like '8k' or '32K tokens'; calling NvidiaNimLLM.promptWindowLimit() (static) before the provider's model sync has overwritten the env var with a numeric max_model_len.","commonSituations":"Human-friendly values in .env; quotes/units copied from a model card; values with trailing spaces in compose files.","solutions":["Set NVIDIA_NIM_LLM_MODEL_TOKEN_LIMIT to a plain integer (e.g. 32768) or remove it to use the 4096 default","Let the provider's model sync populate it from the model's max_model_len by not overriding it manually","Restart the server after fixing .env","Validate numeric env vars at startup"],"exampleFix":"# before\nNVIDIA_NIM_LLM_MODEL_TOKEN_LIMIT=32k\n# after\nNVIDIA_NIM_LLM_MODEL_TOKEN_LIMIT=32768","handlingStrategy":"validation","validationCode":"const raw = process.env.NVIDIA_NIM_LLM_MODEL_TOKEN_LIMIT || '4096';\nconst limit = Number(raw);\nif (!Number.isFinite(limit) || limit <= 0)\n  throw new Error(`NVIDIA_NIM_LLM_MODEL_TOKEN_LIMIT must be an integer, got: ${raw}`);\nconst window = NvidiaNimLLM.promptWindowLimit(modelName);","typeGuard":null,"tryCatchPattern":"try {\n  const limit = NvidiaNimLLM.promptWindowLimit(modelName);\n} catch (e) {\n  if (e.message.includes('token context limit'))\n    return respondConfigError(e); // fix the env value, do not retry\n  throw e;\n}","preventionTips":["Use bare integers for token-limit env vars","Let the provider auto-populate the limit from the model's max_model_len instead of setting it manually","Validate numeric env vars at process startup"],"tags":["nvidia-nim","anythingllm","environment-variables","configuration","token-limit"],"backgroundTag":"invalid-env-var-value","analyzedSha":"3aec848f2885144aa8f1e53b9731a04310d5d558","analyzedAt":"2026-08-18T10:02:21.017Z","contentChangedAt":"2026-08-18T10:02:21.017Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}