Mintplex-Labs/anything-llm · error · Error

No LocalAi token context limit was set.

Error message

No LocalAi token context limit was set.

What it means

Thrown by the static promptWindowLimit on LocalAiLLM when LOCAL_AI_MODEL_TOKEN_LIMIT is set to a truthy but non-numeric value. The `|| 4096` default handles unset/empty values, so this only fires on explicit misconfiguration. The static method is used for pre-flight context-window calculations before a provider instance exists.

Source

Thrown at server/utils/AiProviders/localAi/index.js:51

    if (!contextTexts || !contextTexts.length) return "";
    return (
      "\nContext:\n" +
      contextTexts
        .map((text, i) => {
          return `[CONTEXT ${i}]:\n${text}\n[END CONTEXT ${i}]\n\n`;
        })
        .join("")
    );
  }

  streamingEnabled() {
    return "streamGetChatCompletion" in this;
  }

  static promptWindowLimit(_modelName) {
    const limit = process.env.LOCAL_AI_MODEL_TOKEN_LIMIT || 4096;
    if (!limit || isNaN(Number(limit)))
      throw new Error("No LocalAi token context limit was set.");
    return Number(limit);
  }

  // Ensure the user set a value for the token limit
  // and if undefined - assume 4096 window.
  promptWindowLimit() {
    const limit = process.env.LOCAL_AI_MODEL_TOKEN_LIMIT || 4096;
    if (!limit || isNaN(Number(limit)))
      throw new Error("No LocalAi token context limit was set.");
    return Number(limit);
  }

  async isValidChatCompletionModel(_ = "") {
    return true;
  }

  /**
   * Generates appropriate content array for a message + attachments.

View on GitHub (pinned to 526360e320)

Solutions

1. Set LOCAL_AI_MODEL_TOKEN_LIMIT to a plain integer in .env.
2. Remove the variable to inherit the 4096 default.
3. If using a pre-flight check, catch the error and default to 4096 with a log warning.

Example fix

// before
LOCAL_AI_MODEL_TOKEN_LIMIT='auto'

// after
LOCAL_AI_MODEL_TOKEN_LIMIT=4096

function validateTokenLimit(envVar, defaultLimit = 4096) {
  const raw = process.env[envVar];
  if (!raw) return defaultLimit;
  const num = Number(raw);
  if (isNaN(num)) {
    throw new Error(`${envVar}="${raw}" must be a number or unset for default ${defaultLimit}.`);
  }
  return num;
}

const limit = validateTokenLimit('LOCAL_AI_MODEL_TOKEN_LIMIT');

/** @returns {value is number} */
function isValidTokenLimit(value) {
  return typeof value === 'number' && !isNaN(value) && value > 0;
}

let limit;
try {
  limit = LocalAiLLM.promptWindowLimit();
} catch (e) {
  if (e.message.includes('token context limit')) {
    console.warn('LOCAL_AI_MODEL_TOKEN_LIMIT invalid, defaulting to 4096');
    limit = 4096;
  } else {
    throw e;
  }
}

When it happens

Trigger: Calling `LocalAiLLM.promptWindowLimit(modelName)` when process.env.LOCAL_AI_MODEL_TOKEN_LIMIT is a non-numeric truthy string (e.g. 'auto', 'max', 'default'). Numeric strings pass the isNaN check; unset/empty get the 4096 default.

Common situations: An admin sets a descriptive word instead of a number for the token limit. Copy-pasting a config snippet with a placeholder value. Misunderstanding the variable's expected format.

Related errors

• No token context limit was set.
• No token context limit was set.
• No LocalAI Base Path was set.
• KoboldCPP must have a valid model set.
• No Lemonade API Base Path was set.

AI-assisted analysis of Mintplex-Labs/anything-llm@526360e320 (2026-08-13). Data as JSON: /api/errors/db5400d7e637bf70. Report an issue: GitHub.