Mintplex-Labs/anything-llm · error · Error

TogetherAI chat: ${this.model} is not valid for chat complet

Error message

TogetherAI chat: ${this.model} is not valid for chat completion!

What it means

Thrown by TogetherAiLLM.getChatCompletion when isValidChatCompletionModel(this.model) is false. Together AI's check is stricter than most: it loads the cached model catalog (via togetherAiModels(), persisted to storage/models/togetherai) and requires the id to exist AND have `type === 'chat'`. An embedding, image, or completion-only model is rejected even if the id is valid.

Source

Thrown at server/utils/AiProviders/togetherAi/index.js:183

    attachments = [],
  }) {
    const prompt = {
      role: "system",
      content: `${systemPrompt}${this.#appendContext(contextTexts)}`,
    };
    return [
      prompt,
      ...chatHistory,
      {
        role: "user",
        content: this.#generateContent({ userPrompt, attachments }),
      },
    ];
  }

  async getChatCompletion(messages = null, { temperature = 0.7 }) {
    if (!(await this.isValidChatCompletionModel(this.model)))
      throw new Error(
        `TogetherAI chat: ${this.model} is not valid for chat completion!`
      );

    const result = await LLMPerformanceMonitor.measureAsyncFunction(
      this.openai.chat.completions
        .create({
          model: this.model,
          messages,
          temperature,
        })
        .catch((e) => {
          throw new Error(e.message);
        })
    );

    if (
      !result.output.hasOwnProperty("choices") ||
      result.output.choices.length === 0

View on GitHub (pinned to 526360e320)

Solutions

1. Check storage/models/togetherai for the catalog and confirm the model id is present with type 'chat'; re-fetch via Together AI's /models endpoint if stale.
2. Switch TOGETHER_AI_MODEL_PREF to a chat model (e.g. meta-llama/Llama-3-70b-chat-hf).
3. Delete the stale cache so the next togetherAiModels() call repopulates it.
4. Re-select a chat model in the AnythingLLM UI.

Example fix

// before
const llm = new TogetherAiLLM(embedder, "BAAI/bge-large-en-v1.5"); // type=embedding -> rejects
await llm.getChatCompletion(messages, { temperature: 0.7 });

// after
const models = await togetherAiModels();
const chat = models.find((m) => m.id === "meta-llama/Llama-3-70b-chat-hf" && m.type === "chat");
const id = chat ? chat.id : "meta-llama/Llama-3-70b-chat-hf";
const llm2 = new TogetherAiLLM(embedder, id);
await llm2.getChatCompletion(messages, { temperature: 0.7 });

const models = await togetherAiModels();
function pickTogetherChatModel(pref) {
  const m = models.find((x) => x.id === pref && x.type === "chat");
  if (m) return m.id;
  const any = models.find((x) => x.type === "chat");
  if (any) return any.id;
  throw new Error("Together AI catalog has no chat models — refresh storage/models/togetherai");
}
const id = pickTogetherChatModel(process.env.TOGETHER_AI_MODEL_PREF);
const llm = new TogetherAiLLM(embedder, id);

function isTogetherChatModel(id, models) {
  const m = Array.isArray(models) ? models.find((x) => x.id === id) : null;
  return !!m && m.type === "chat";
}

try {
  return await llm.getChatCompletion(messages, opts);
} catch (e) {
  if (/not valid for chat completion/i.test(e.message)) {
    const models = await togetherAiModels();
    const chat = models.find((m) => m.type === "chat");
    if (!chat) throw e;
    llm.model = chat.id;
    return llm.getChatCompletion(messages, opts);
  }
  throw e;
}

When it happens

Trigger: this.model is missing from the cached catalog, OR present but with a type other than 'chat' (e.g. an embedding model id like 'BAAI/bge-large-en-v1.5' or a code-completion-only model). Cache is empty/stale if togetherAiModels() failed to fetch.

Common situations: TOGETHER_AI_MODEL_PREF set to an embedding/finetune/image model by mistake; the cached model list at storage/models/togetherai is stale after Together AI added/removed models; the catalog fetch (which hits Together's /models) failed at first run and cached nothing; copy-pasting an OpenAI id.

Related errors

• Perplexity chat: ${this.model} is not valid for chat complet
• PPIO chat: ${this.model} is not valid for chat completion!
• Privatemode chat: ${this.model} is not valid or defined mode
• CometAPI chat: ${this.model} is not valid for chat completio
• FireworksAI chat: ${this.model} is not valid for chat comple

AI-assisted analysis of Mintplex-Labs/anything-llm@526360e320 (2026-08-13). Data as JSON: /api/errors/f021360e4f037a87. Report an issue: GitHub.