Mintplex-Labs/anything-llm · error · Error

Foundry chat: is not valid or defined model for chat…

Error message

Foundry chat: ${this.model} is not valid or defined model for chat completion!

What it means

FoundryLLM.getChatCompletion throws when this.model is falsy before it can resolve the context window (max_completion_tokens is mandatory for Foundry) and load the model. The constructor sets this.model from modelPreference || FOUNDRY_MODEL_PREF, both of which are optional, so unlike Docker Model Runner there is no constructor-time guard — an empty model survives construction and fails here.

Solutions

1. Set FOUNDRY_MODEL_PREF to a valid model id (e.g. ai/qwen2.5-0.5b-instruct) in .env and restart
2. Or select a specific Foundry model in the workspace's model picker so modelPreference is non-empty
3. If constructing programmatically, pass a non-empty modelPreference second argument

Example fix

# before (.env)
FOUNDRY_BASE_PATH=http://localhost:5273/v1
# no model pref -> throws at chat time

# after (.env)
FOUNDRY_BASE_PATH=http://localhost:5273/v1
FOUNDRY_MODEL_PREF=ai/qwen2.5-0.5b-instruct

const model = modelPreference || process.env.FOUNDRY_MODEL_PREF;
if (!model) {
  return respond("Select a Foundry Local model before sending a message.");
}
const llm = new FoundryLLM(embedder, model);

try {
  await llm.getChatCompletion(messages);
} catch (err) {
  if (/not valid or defined model/.test(err.message)) {
    return respond("No Foundry model configured — set FOUNDRY_MODEL_PREF or pick one.");
  }
  throw err;
}

When it happens

Trigger: Any non-streaming chat request when neither a workspace modelPreference nor FOUNDRY_MODEL_PREF was provided. Constructing the provider succeeds (env path present, model blank), then the first getChatCompletion trips this guard.

Common situations: FOUNDRY_BASE_PATH configured but FOUNDRY_MODEL_PREF never set and the workspace model selection left empty; the UI passed an empty string as the preference; env var name typo like FOUNDRY_MODEL.

Related errors

• Could not load into Foundry Local
• e.message
• FireworksAI chat: is not valid for chat completion!
• Foundry could not load
• GenericOpenAI must have a valid model set.

AI-assisted analysis of Mintplex-Labs/anything-llm@3aec848f28 (2026-08-18). Data as JSON: /api/errors/a0c363a51f6769e1. Report an issue: GitHub.

Appendix: source

Thrown at server/utils/AiProviders/foundry/index.js:307

    attachments = [],
  }) {
    const prompt = {
      role: "system",
      content: `${systemPrompt}${this.#appendContext(contextTexts)}`,
    };
    return [
      prompt,
      ...formatChatHistory(chatHistory, this.#generateContent),
      {
        role: "user",
        content: this.#generateContent({ userPrompt, attachments }),
      },
    ];
  }

  async getChatCompletion(messages = null, { temperature = 0.7 }) {
    if (!this.model)
      throw new Error(
        `Foundry chat: ${this.model} is not valid or defined model for chat completion!`
      );

    // max_completion_tokens is required by Foundry (it caps output at 1024
    // otherwise), so the window has to be resolved before the request is built.
    await this.assertModelContextLimits();
    await this.assertModelLoaded();
    const result = await LLMPerformanceMonitor.measureAsyncFunction(
      this.openai.chat.completions
        .create({
          model: this.model,
          messages,
          temperature,
          max_completion_tokens: this.promptWindowLimit(),
        })
        .catch((e) => {
          throw new Error(e.message);
        })

View on GitHub (pinned to 3aec848f28)