Mintplex-Labs/anything-llm · error · Error

GroqAI:streamChatCompletion

Error message

GroqAI:streamChatCompletion: ${this.model} is not valid for chat completion!

What it means

Same falsy-model guard as the non-stream path, but at the top of GroqLLM.streamGetChatCompletion(). isValidChatCompletionModel only checks the name is non-empty (`!!modelName`), so this fires only when this.model is empty/null right before streaming. The constructor's fallback chain (modelPreference || GROQ_MODEL_PREF || "llama-3.1-8b-instant") makes that a defensive, near-unreachable branch.

Solutions

1. Log the instance's resolved model (llm.model) before streaming; if empty, set GROQ_MODEL_PREF or pass modelPreference at construction.
2. Pin GROQ_MODEL_PREF to a live model id from console.groq.com/models so the default chain never resolves empty.
3. If streaming actually fails mid-request, debug the underlying SDK error from measureStream instead of this guard.

Example fix

// before
GROQ_MODEL_PREF=            # blank in .env

// after
GROQ_MODEL_PREF=llama-3.3-70b-versatile

if (!llm.model) throw new Error("GroqLLM has no model — reconstruct with modelPreference or set GROQ_MODEL_PREF");
await llm.streamGetChatCompletion(messages);

function isStreamableGroqInstance(llm) {
  return llm instanceof GroqLLM && typeof llm.model === "string" && llm.model.length > 0;
}

try {
  await llm.streamGetChatCompletion(messages);
} catch (e) {
  if (/not valid for chat completion/i.test(e.message)) rebuildProviderWithModel();
  else throw e;
}

When it happens

Trigger: Invoking llm.streamGetChatCompletion(messages) (i.e., streaming a chat response with Groq selected) on an instance whose model resolved to an empty string — only realistic with a mutated instance or a patched constructor default that bypasses the fallback.

Common situations: Effectively never hit in normal AnythingLLM use; appears in stack traces only as a misdiagnosis when the actual streaming failure is an HTTP-level error from the Groq API (bad key, decommissioned model, rate limit) thrown later inside LLMPerformanceMonitor.measureStream.

Related errors

• GroqAI:chatCompletion
• data.message || "Could not update slash command preset."
• data.message || "Error creating slash command preset."
• e.message
• Error downloading model

AI-assisted analysis of Mintplex-Labs/anything-llm@f92433b4ea (2026-08-18). Data as JSON: /api/errors/a99b787b3bfd7c8a. Report an issue: GitHub.

Appendix: source

Thrown at server/utils/AiProviders/groq/index.js:212

      textResponse: result.output.choices[0].message.content,
      metrics: {
        prompt_tokens: result.output.usage.prompt_tokens || 0,
        completion_tokens: result.output.usage.completion_tokens || 0,
        total_tokens: result.output.usage.total_tokens || 0,
        outputTps:
          result.output.usage.completion_tokens /
          result.output.usage.completion_time,
        duration: result.output.usage.total_time,
        model: this.model,
        provider: this.className,
        timestamp: new Date(),
      },
    };
  }

  async streamGetChatCompletion(messages = null, { temperature = 0.7 }) {
    if (!(await this.isValidChatCompletionModel(this.model)))
      throw new Error(
        `GroqAI:streamChatCompletion: ${this.model} is not valid for chat completion!`
      );

    const measuredStreamRequest = await LLMPerformanceMonitor.measureStream({
      func: this.openai.chat.completions.create({
        model: this.model,
        stream: true,
        messages,
        temperature,
      }),
      messages,
      runPromptTokenCalculation: false,
      modelTag: this.model,
      provider: this.className,
    });

    return measuredStreamRequest;
  }

View on GitHub (pinned to f92433b4ea)