Mintplex-Labs/anything-llm · error · Error

No token context limit was set.

Error message

No token context limit was set.

What it means

Thrown by the static promptWindowLimit on LiteLLM when LITE_LLM_MODEL_TOKEN_LIMIT is set to a truthy but non-numeric value. The `|| 4096` default absorbs unset/empty values, so this only fires on explicit misconfiguration like setting it to 'auto' or 'none'. The static method is used for pre-flight context calculations.

Source

Thrown at server/utils/AiProviders/liteLLM/index.js:61

    if (!contextTexts || !contextTexts.length) return "";
    return (
      "\nContext:\n" +
      contextTexts
        .map((text, i) => {
          return `[CONTEXT ${i}]:\n${text}\n[END CONTEXT ${i}]\n\n`;
        })
        .join("")
    );
  }

  streamingEnabled() {
    return "streamGetChatCompletion" in this;
  }

  static promptWindowLimit(_modelName) {
    const limit = process.env.LITE_LLM_MODEL_TOKEN_LIMIT || 4096;
    if (!limit || isNaN(Number(limit)))
      throw new Error("No token context limit was set.");
    return Number(limit);
  }

  // Ensure the user set a value for the token limit
  // and if undefined - assume 4096 window.
  promptWindowLimit() {
    const limit = process.env.LITE_LLM_MODEL_TOKEN_LIMIT || 4096;
    if (!limit || isNaN(Number(limit)))
      throw new Error("No token context limit was set.");
    return Number(limit);
  }

  // Short circuit since we have no idea if the model is valid or not
  // in pre-flight for generic endpoints
  isValidChatCompletionModel(_modelName = "") {
    return true;
  }

View on GitHub (pinned to 526360e320)

Solutions

1. Set LITE_LLM_MODEL_TOKEN_LIMIT to a plain integer in .env.
2. Remove the variable to use the default 4096.
3. If you need different limits per model, set the variable dynamically or handle it in the calling code.

Example fix

// before
LITE_LLM_MODEL_TOKEN_LIMIT='max'

// after
LITE_LLM_MODEL_TOKEN_LIMIT=8192

function validateTokenLimit(envVar, defaultLimit = 4096) {
  const raw = process.env[envVar];
  if (!raw) return defaultLimit;
  const num = Number(raw);
  if (isNaN(num)) {
    throw new Error(`${envVar}="${raw}" must be a number or unset for default ${defaultLimit}.`);
  }
  return num;
}

const limit = validateTokenLimit('LITE_LLM_MODEL_TOKEN_LIMIT');

/** @returns {value is number} */
function isValidTokenLimit(value) {
  return typeof value === 'number' && !isNaN(value) && value > 0;
}

let limit;
try {
  limit = LiteLLM.promptWindowLimit();
} catch (e) {
  if (e.message.includes('token context limit')) {
    console.warn('LITE_LLM_MODEL_TOKEN_LIMIT invalid, defaulting to 4096');
    limit = 4096;
  } else {
    throw e;
  }
}

When it happens

Trigger: Calling `LiteLLM.promptWindowLimit(modelName)` when process.env.LITE_LLM_MODEL_TOKEN_LIMIT is a non-numeric string. Numeric strings, unset, and empty are handled by the default or pass the isNaN check.

Common situations: An admin sets the token limit to a descriptive word thinking it's a label. Copy-pasting a configuration snippet from documentation that uses a placeholder. Setting the value to a float-formatted string like '4.096e3' which actually works (Number('4.096e3') = 4096) but is confusing.

Related errors

• No token context limit was set.
• No LocalAi token context limit was set.
• LiteLLM must have a valid base path to use for the api.
• LiteLLM must have a valid model set.
• KoboldCPP must have a valid model set.

AI-assisted analysis of Mintplex-Labs/anything-llm@526360e320 (2026-08-13). Data as JSON: /api/errors/bd6807448fb594b0. Report an issue: GitHub.