Mintplex-Labs/anything-llm · error · Error

${e.message}

Error message

${e.message}

What it means

Re-thrown from the OpenAI SDK rejection inside TogetherAiLLM.getChatCompletion via `.catch((e) => { throw new Error(e.message); })`. The wrapper discards the SDK error class and HTTP status, so a 401 (auth), 429 (Together AI free-tier rate limits are common), and a server error look alike to the caller except for the message text.

Source

Thrown at server/utils/AiProviders/togetherAi/index.js:195

      },
    ];
  }

  async getChatCompletion(messages = null, { temperature = 0.7 }) {
    if (!(await this.isValidChatCompletionModel(this.model)))
      throw new Error(
        `TogetherAI chat: ${this.model} is not valid for chat completion!`
      );

    const result = await LLMPerformanceMonitor.measureAsyncFunction(
      this.openai.chat.completions
        .create({
          model: this.model,
          messages,
          temperature,
        })
        .catch((e) => {
          throw new Error(e.message);
        })
    );

    if (
      !result.output.hasOwnProperty("choices") ||
      result.output.choices.length === 0
    )
      return null;

    return {
      textResponse: result.output.choices[0].message.content,
      metrics: {
        prompt_tokens: result.output.usage?.prompt_tokens || 0,
        completion_tokens: result.output.usage?.completion_tokens || 0,
        total_tokens: result.output.usage?.total_tokens || 0,
        outputTps: result.output.usage?.completion_tokens / result.duration,
        duration: result.duration,
        model: this.model,

View on GitHub (pinned to 526360e320)

Solutions

1. Decode the message: '401'/'Unauthorized' -> key; '429'/'rate limit' -> throttle or upgrade tier; 'model'/'not found' -> model id; 'timeout'/'ECONN' -> network.
2. curl POST https://api.together.xyz/v1/chat/completions with the same key/model to isolate.
3. For 429, add bounded exponential-backoff retry and/or reduce concurrency.
4. Check the Together AI dashboard for quota/usage and model availability.

Example fix

// before
const out = await llm.getChatCompletion(messages, { temperature: 0.7 });

// after
async function togetherWithBackoff(llm, messages, opts, retries = 4) {
  for (let i = 0; i <= retries; i++) {
    try {
      return await llm.getChatCompletion(messages, opts);
    } catch (e) {
      if (!/429|5\d{2}|timeout|ECONN/i.test(e.message) || i === retries) throw e;
      await new Promise((r) => setTimeout(r, 2 ** i * 500));
    }
  }
}
const out = await togetherWithBackoff(llm, messages, { temperature: 0.7 });

function classifyTogetherError(message) {
  if (/401|unauthorized|invalid api key/i.test(message)) return "auth";
  if (/model|not found/i.test(message)) return "model";
  if (/429|rate limit|quota/i.test(message)) return "rate"; // Together free tier is aggressive
  if (/5\d{2}|timeout|ECONN/i.test(message)) return "transient";
  return "fatal";
}

function isTransientTogetherError(message) {
  return typeof message === "string" && /429|5\d{2}|timeout|ECONNRESET|fetch failed/i.test(message);
}

async function togetherCall(llm, messages, opts, retries = 4) {
  for (let i = 0; i <= retries; i++) {
    try {
      return await llm.getChatCompletion(messages, opts);
    } catch (e) {
      const kind = classifyTogetherError(e.message);
      if (kind === "auth" || kind === "model" || i === retries) throw e;
      await new Promise((r) => setTimeout(r, 2 ** i * 500)); // backoff for rate/transient
    }
  }
}

When it happens

Trigger: api.together.xyz returns non-2xx or the client throws pre-response: 401 invalid/expired key, 404 model not deployed, 429 rate/quota (Together's free tier is aggressive), 5xx, DNS/TLS/abort errors.

Common situations: Free-tier Together key hitting the per-minute request cap; key deactivated after startup; TOGETHER_AI_MODEL_PREF pointed at a model that passed local cache validation but is not enabled for the key; flaky network egress; client aborted the streaming request.

Related errors

• ${e.message}
• ${e.message}
• ${e.message}
• ${e.message}
• e.message

AI-assisted analysis of Mintplex-Labs/anything-llm@526360e320 (2026-08-13). Data as JSON: /api/errors/67a0cbd1f9f7c1f3. Report an issue: GitHub.