Mintplex-Labs/anything-llm · error

No token context limit was set.

Error message

No token context limit was set.

What it means

Static KoboldCPPLLM.promptWindowLimit(_modelName) reads KOBOLD_CPP_MODEL_TOKEN_LIMIT with a 4096 default and throws 'No token context limit was set.' only if the value is falsy or Number(limit) is NaN. Because of the `|| 4096` fallback the falsy branch is dead code; the only live trigger is the env var being set to a non-numeric string. The static variant is used by AnythingLLM's provider-selection code to compute context windows before an instance exists.

Solutions

1. Set the variable to a plain integer: KOBOLD_CPP_MODEL_TOKEN_LIMIT=8192.
2. Remove commas, units, quotes, and inline comments from the value.
3. Or simply unset/delete the variable — the code falls back to 4096.
4. Sanitize at deploy time: assert /^[0-9]+$/ against the var before starting the server.

Example fix

# before (.env)
KOBOLD_CPP_MODEL_TOKEN_LIMIT=16,384

# after (.env)
KOBOLD_CPP_MODEL_TOKEN_LIMIT=16384

const raw = process.env.KOBOLD_CPP_MODEL_TOKEN_LIMIT;
if (raw !== undefined && raw !== "" && !/^\d+$/.test(raw.trim())) {
  throw new Error(`KOBOLD_CPP_MODEL_TOKEN_LIMIT must be a bare integer, got: ${raw}`);
}
const window = KoboldCPPLLM.promptWindowLimit(modelName);

function isParsableTokenLimit(value) {
  return value !== undefined && value !== null && value !== "" && !isNaN(Number(value));
}

When it happens

Trigger: Calling KoboldCPPLLM.promptWindowLimit(modelName) (statically, during provider capability lookup) while KOBOLD_CPP_MODEL_TOKEN_LIMIT contains a non-numeric value such as '4,096', '4096 tokens', '16k', or a stray quote. The NaN check converts it into this throw.

Common situations: Users pasting a formatted number with a thousands separator or unit into .env; a trailing space plus quote ("4096") from copy-paste; accidentally documenting the var inline (4096 # context) which JS Number() cannot parse; CI injecting the var as 'null' or 'undefined' from a template.

Understand the failure class

Background: "is not a valid" / "Invalid ... value" environment variable errors: how libraries validate env vars and what to do when they reject yours — this error's family across 48 libraries.

Related errors

• No LocalAi token context limit was set.
• No token context limit was set.
• KoboldCPP must have a valid base path to use for the api.
• KoboldCPP must have a valid model set.
• LiteLLM must have a valid base path to use for the api.

AI-assisted analysis of Mintplex-Labs/anything-llm@3aec848f28 (2026-08-18). Data as JSON: /api/errors/69da8482af23ccbb. Report an issue: GitHub.

Appendix: source

Thrown at server/utils/AiProviders/koboldCPP/index.js:63

    if (!contextTexts || !contextTexts.length) return "";
    return (
      "\nContext:\n" +
      contextTexts
        .map((text, i) => {
          return `[CONTEXT ${i}]:\n${text}\n[END CONTEXT ${i}]\n\n`;
        })
        .join("")
    );
  }

  streamingEnabled() {
    return "streamGetChatCompletion" in this;
  }

  static promptWindowLimit(_modelName) {
    const limit = process.env.KOBOLD_CPP_MODEL_TOKEN_LIMIT || 4096;
    if (!limit || isNaN(Number(limit)))
      throw new Error("No token context limit was set.");
    return Number(limit);
  }

  // Ensure the user set a value for the token limit
  // and if undefined - assume 4096 window.
  promptWindowLimit() {
    const limit = process.env.KOBOLD_CPP_MODEL_TOKEN_LIMIT || 4096;
    if (!limit || isNaN(Number(limit)))
      throw new Error("No token context limit was set.");
    return Number(limit);
  }

  // Short circuit since we have no idea if the model is valid or not
  // in pre-flight for generic endpoints
  isValidChatCompletionModel(_modelName = "") {
    return true;
  }

View on GitHub (pinned to 3aec848f28)