Mintplex-Labs/anything-llm · error · Error

No token context limit was set.

Error message

No token context limit was set.

What it means

Thrown by the static promptWindowLimit method on KoboldCPPLLM. It reads KOBOLD_CPP_MODEL_TOKEN_LIMIT, defaulting to 4096 via `||`. The error fires only when the env var is set to a truthy but non-numeric value (e.g. 'unlimited'), because the `|| 4096` fallback already absorbs unset/empty/falsy values. The static variant is used for pre-flight context-window calculations before a provider instance exists.

Source

Thrown at server/utils/AiProviders/koboldCPP/index.js:63

    if (!contextTexts || !contextTexts.length) return "";
    return (
      "\nContext:\n" +
      contextTexts
        .map((text, i) => {
          return `[CONTEXT ${i}]:\n${text}\n[END CONTEXT ${i}]\n\n`;
        })
        .join("")
    );
  }

  streamingEnabled() {
    return "streamGetChatCompletion" in this;
  }

  static promptWindowLimit(_modelName) {
    const limit = process.env.KOBOLD_CPP_MODEL_TOKEN_LIMIT || 4096;
    if (!limit || isNaN(Number(limit)))
      throw new Error("No token context limit was set.");
    return Number(limit);
  }

  // Ensure the user set a value for the token limit
  // and if undefined - assume 4096 window.
  promptWindowLimit() {
    const limit = process.env.KOBOLD_CPP_MODEL_TOKEN_LIMIT || 4096;
    if (!limit || isNaN(Number(limit)))
      throw new Error("No token context limit was set.");
    return Number(limit);
  }

  // Short circuit since we have no idea if the model is valid or not
  // in pre-flight for generic endpoints
  isValidChatCompletionModel(_modelName = "") {
    return true;
  }

View on GitHub (pinned to 526360e320)

Solutions

1. Set KOBOLD_CPP_MODEL_TOKEN_LIMIT to a numeric value (e.g. 4096, 8192, 16384) in .env.
2. Remove or comment out KOBOLD_CPP_MODEL_TOKEN_LIMIT entirely to fall back to the built-in 4096 default.
3. If using a pre-flight context-size check, catch the error and default to 4096 with a warning log.

Example fix

// before
KOBOLD_CPP_MODEL_TOKEN_LIMIT='unlimited'

// after
KOBOLD_CPP_MODEL_TOKEN_LIMIT=8192

function validateTokenLimit(envVar, defaultLimit = 4096) {
  const raw = process.env[envVar];
  if (!raw) return defaultLimit;
  const num = Number(raw);
  if (isNaN(num)) {
    throw new Error(
      `${envVar}="${raw}" is not a number. Set it to an integer or unset it for default ${defaultLimit}.`
    );
  }
  return num;
}

// Pre-flight usage:
const limit = validateTokenLimit('KOBOLD_CPP_MODEL_TOKEN_LIMIT');

/** @returns {value is number} */
function isValidTokenLimit(value) {
  return typeof value === 'number' && !isNaN(value) && value > 0;
}

let limit;
try {
  limit = KoboldCPPLLM.promptWindowLimit();
} catch (e) {
  if (e.message.includes('token context limit')) {
    console.warn('KOBOLD_CPP_MODEL_TOKEN_LIMIT is invalid, defaulting to 4096');
    limit = 4096;
  } else {
    throw e;
  }
}

When it happens

Trigger: Calling `KoboldCPPLLM.promptWindowLimit(modelName)` when process.env.KOBOLD_CPP_MODEL_TOKEN_LIMIT is set to a non-numeric string such as 'auto', 'unlimited', 'max', or 'n/a'. Numeric strings like '8192' or '0' do NOT trigger it (0 passes isNaN but yields zero-length limits).

Common situations: A user copies a token-limit value from a forum or blog that uses a word instead of a number. Setting the variable to 'unlimited' expecting KoboldCPP to auto-detect context size. Misunderstanding the comment 'if undefined - assume 4096' and setting a placeholder word.

Related errors

• No token context limit was set.
• No LocalAi token context limit was set.
• KoboldCPP must have a valid model set.
• KoboldCPP must have a valid base path to use for the api.
• No Lemonade API Base Path was set.

AI-assisted analysis of Mintplex-Labs/anything-llm@526360e320 (2026-08-13). Data as JSON: /api/errors/69da8482af23ccbb. Report an issue: GitHub.