Mintplex-Labs/anything-llm · error · Error

No token context limit was set.

Error message

No token context limit was set.

What it means

Thrown by the static promptWindowLimit(_modelName) when the resolved limit is falsy or non-numeric. Because of the `|| 4096` fallback an unset var yields 4096 (valid), so in practice this fires only when GENERIC_OPEN_AI_MODEL_TOKEN_LIMIT is set to a non-numeric string such as 'unlimited' or '4k'. The static variant is used before an instance exists.

Source

Thrown at server/utils/AiProviders/genericOpenAi/index.js:102

    return (
      "\nContext:\n" +
      contextTexts
        .map((text, i) => {
          return `[CONTEXT ${i}]:\n${text}\n[END CONTEXT ${i}]\n\n`;
        })
        .join("")
    );
  }

  streamingEnabled() {
    if (process.env.GENERIC_OPENAI_STREAMING_DISABLED === "true") return false;
    return "streamGetChatCompletion" in this;
  }

  static promptWindowLimit(_modelName) {
    const limit = process.env.GENERIC_OPEN_AI_MODEL_TOKEN_LIMIT || 4096;
    if (!limit || isNaN(Number(limit)))
      throw new Error("No token context limit was set.");
    return Number(limit);
  }

  // Ensure the user set a value for the token limit
  // and if undefined - assume 4096 window.
  promptWindowLimit() {
    const limit = process.env.GENERIC_OPEN_AI_MODEL_TOKEN_LIMIT || 4096;
    if (!limit || isNaN(Number(limit)))
      throw new Error("No token context limit was set.");
    return Number(limit);
  }

  // Short circuit since we have no idea if the model is valid or not
  // in pre-flight for generic endpoints
  isValidChatCompletionModel(_modelName = "") {
    return true;
  }

View on GitHub (pinned to 526360e320)

Solutions

1. Set GENERIC_OPEN_AI_MODEL_TOKEN_LIMIT to a plain integer, e.g. 8192
2. Leave it unset to accept the 4096 default
3. Add UI/env validation to reject non-numeric token limits
4. After fixing, restart the server

Example fix

// before
// .env: GENERIC_OPEN_AI_MODEL_TOKEN_LIMIT=8k

// after
// .env: GENERIC_OPEN_AI_MODEL_TOKEN_LIMIT=8192

function validTokenLimit() {
  const raw = process.env.GENERIC_OPEN_AI_MODEL_TOKEN_LIMIT;
  if (raw == null || raw === '') return 4096;
  const n = Number(raw);
  if (Number.isNaN(n)) {
    throw new ConfigError('GENERIC_OPEN_AI_MODEL_TOKEN_LIMIT must be numeric.');
  }
  return n;
}

const isTokenLimit = (v) =>
  v == null || v === '' ? true : (typeof v === 'string' || typeof v === 'number') && !Number.isNaN(Number(v));

When it happens

Trigger: Calling GenericOpenAiLLM.promptWindowLimit(modelName) (static) while GENERIC_OPEN_AI_MODEL_TOKEN_LIMIT is a non-numeric, non-empty value like 'large' or 'auto'. An empty/undefined value is safe (defaults to 4096).

Common situations: User typed a human word instead of a number in the token-limit field; copy-pasted a config with units ('8k'); UI validation not enforcing numeric input.

Related errors

• GenericOpenAI must have a valid base path to use for the api
• GenericOpenAI must have a valid model set.
• No NVIDIA NIM token context limit was set.
• OMLX must have a valid model set.
• OpenAI chat: ${this.model} is not valid for chat completion!

AI-assisted analysis of Mintplex-Labs/anything-llm@526360e320 (2026-08-13). Data as JSON: /api/errors/e0f6e6afcf75405c. Report an issue: GitHub.