Mintplex-Labs/anything-llm · error · Error

No token context limit was set.

Error message

No token context limit was set.

What it means

The static promptWindowLimit reads GENERIC_OPEN_AI_MODEL_TOKEN_LIMIT (defaulting the value to 4096 when the env var is empty/unset) and throws when Number(limit) is NaN — i.e. the env var is set to a non-numeric string. Because the || fallback only substitutes for falsy values, a garbage non-empty value like '4k' or '8192 ' passes into the NaN check and throws. This static variant runs during provider-selection/class-level queries such as context-window sizing before an instance exists.

Solutions

1. Set GENERIC_OPEN_AI_MODEL_TOKEN_LIMIT to a bare integer, e.g. 8192 (no 'k', commas, or units), and restart
2. Or remove the variable entirely — the code then defaults to 4096
3. Check for invisible characters/quotes: print it with node -e "console.log(JSON.stringify(process.env.GENERIC_OPEN_AI_MODEL_TOKEN_LIMIT))"

Example fix

# before (.env)
GENERIC_OPEN_AI_MODEL_TOKEN_LIMIT=128k

# after (.env)
GENERIC_OPEN_AI_MODEL_TOKEN_LIMIT=131072

const raw = process.env.GENERIC_OPEN_AI_MODEL_TOKEN_LIMIT;
if (raw != null && raw !== "" && Number.isNaN(Number(raw))) {
  throw new Error(`GENERIC_OPEN_AI_MODEL_TOKEN_LIMIT must be numeric, got '${raw}'`);
}
// safe to call GenericOpenAiLLM.promptWindowLimit(modelName)

try {
  const limit = GenericOpenAiLLM.promptWindowLimit(modelName);
} catch (err) {
  if (err.message === "No token context limit was set.") {
    return respond("GENERIC_OPEN_AI_MODEL_TOKEN_LIMIT must be a plain integer like 8192.");
  }
  throw err;
}

When it happens

Trigger: GENERIC_OPEN_AI_MODEL_TOKEN_LIMIT set to something non-numeric: '4k', '16k tokens', '8,192', or a value with stray characters. Note: whitespace-only or numeric-with-comma strings all produce NaN; plain '0' or '' do NOT throw (they fall back or return 0).

Common situations: User copies a token limit styled like marketing copy ('128k context') into the env var; values pasted with commas or units; shell quoting artifacts around the number.

Related errors

• GenericOpenAI must have a valid base path to use for the…
• No LocalAi token context limit was set.
• No NVIDIA NIM token context limit was set.
• No token context limit was set.
• No token context limit was set.

AI-assisted analysis of Mintplex-Labs/anything-llm@3aec848f28 (2026-08-18). Data as JSON: /api/errors/e0f6e6afcf75405c. Report an issue: GitHub.

Appendix: source

Thrown at server/utils/AiProviders/genericOpenAi/index.js:102

    return (
      "\nContext:\n" +
      contextTexts
        .map((text, i) => {
          return `[CONTEXT ${i}]:\n${text}\n[END CONTEXT ${i}]\n\n`;
        })
        .join("")
    );
  }

  streamingEnabled() {
    if (process.env.GENERIC_OPENAI_STREAMING_DISABLED === "true") return false;
    return "streamGetChatCompletion" in this;
  }

  static promptWindowLimit(_modelName) {
    const limit = process.env.GENERIC_OPEN_AI_MODEL_TOKEN_LIMIT || 4096;
    if (!limit || isNaN(Number(limit)))
      throw new Error("No token context limit was set.");
    return Number(limit);
  }

  // Ensure the user set a value for the token limit
  // and if undefined - assume 4096 window.
  promptWindowLimit() {
    const limit = process.env.GENERIC_OPEN_AI_MODEL_TOKEN_LIMIT || 4096;
    if (!limit || isNaN(Number(limit)))
      throw new Error("No token context limit was set.");
    return Number(limit);
  }

  // Short circuit since we have no idea if the model is valid or not
  // in pre-flight for generic endpoints
  isValidChatCompletionModel(_modelName = "") {
    return true;
  }

View on GitHub (pinned to 3aec848f28)