Mintplex-Labs/anything-llm · error

No LocalAi token context limit was set.

Error message

No LocalAi token context limit was set.

What it means

LocalAiLLM.promptWindowLimit(): `const limit = process.env.LOCAL_AI_MODEL_TOKEN_LIMIT || 4096; if (!limit || isNaN(Number(limit))) throw 'No LocalAi token context limit was set.'`. As with the KoboldCPP/LiteLLM twins, the || 4096 fallback kills the falsy branch — the throw means the env var is set to a string Number() cannot parse. The constructor calls it to compute this.limits, so it fails at provider construction time.

Solutions

1. Set LOCAL_AI_MODEL_TOKEN_LIMIT to a bare integer (e.g., 16384) or remove it to use the 4096 default.
2. Sanitize the .env: no commas, units, quotes, spaces, inline comments; verify with cat -A.
3. In Docker, ensure the variable is declared with a clean value under environment: and recreate the container.
4. Add a boot-time validation step that Number()-parses all *_TOKEN_LIMIT vars and fails fast with a clear message.

Example fix

# before (.env)
LOCAL_AI_MODEL_TOKEN_LIMIT=16,384

# after (.env)
LOCAL_AI_MODEL_TOKEN_LIMIT=16384

const raw = process.env.LOCAL_AI_MODEL_TOKEN_LIMIT;
if (raw && !/^\d+$/.test(raw.trim())) throw new Error(`LOCAL_AI_MODEL_TOKEN_LIMIT must be an integer, got ${JSON.stringify(raw)}`);
const llm = new LocalAiLLM(embedder, model);

const isParsableLimitEnv = (v) => v == null || v === "" || /^\d+$/.test(String(v).trim());

try {
  const llm = new LocalAiLLM(embedder, model);
} catch (e) {
  if (/LocalAi token context limit/i.test(e.message)) throw new Error("Fix LOCAL_AI_MODEL_TOKEN_LIMIT to a bare integer or unset for the 4096 default");
  throw e;
}

When it happens

Trigger: Instantiating LocalAiLLM (i.e., choosing LocalAI as provider) or calling promptWindowLimit() while LOCAL_AI_MODEL_TOKEN_LIMIT holds a non-numeric value — '16384 ', '16k', '16,384', quoted numbers, or CRLF-polluted lines from Windows-edited .env files.

Common situations: Pasting context-window specs with units; dotenv files with BOM/CRLF artifacts; compose interpolation injecting empty-but-present values (${VAR} with VAR unsets can yield ''); secret managers storing numbers as quoted strings.

Understand the failure class

Background: "is not a valid" / "Invalid ... value" environment variable errors: how libraries validate env vars and what to do when they reject yours — this error's family across 48 libraries.

Related errors

• No token context limit was set.
• No token context limit was set.
• No LocalAI Base Path was set.
• KoboldCPP must have a valid base path to use for the api.
• KoboldCPP must have a valid model set.

AI-assisted analysis of Mintplex-Labs/anything-llm@3aec848f28 (2026-08-18). Data as JSON: /api/errors/db5400d7e637bf70. Report an issue: GitHub.

Appendix: source

Thrown at server/utils/AiProviders/localAi/index.js:51

    if (!contextTexts || !contextTexts.length) return "";
    return (
      "\nContext:\n" +
      contextTexts
        .map((text, i) => {
          return `[CONTEXT ${i}]:\n${text}\n[END CONTEXT ${i}]\n\n`;
        })
        .join("")
    );
  }

  streamingEnabled() {
    return "streamGetChatCompletion" in this;
  }

  static promptWindowLimit(_modelName) {
    const limit = process.env.LOCAL_AI_MODEL_TOKEN_LIMIT || 4096;
    if (!limit || isNaN(Number(limit)))
      throw new Error("No LocalAi token context limit was set.");
    return Number(limit);
  }

  // Ensure the user set a value for the token limit
  // and if undefined - assume 4096 window.
  promptWindowLimit() {
    const limit = process.env.LOCAL_AI_MODEL_TOKEN_LIMIT || 4096;
    if (!limit || isNaN(Number(limit)))
      throw new Error("No LocalAi token context limit was set.");
    return Number(limit);
  }

  async isValidChatCompletionModel(_ = "") {
    return true;
  }

  /**
   * Generates appropriate content array for a message + attachments.

View on GitHub (pinned to 3aec848f28)