chroma-core/chroma · error · Error

Invalid response format from Together AI API

Error message

Invalid response format from Together AI API

What it means

Thrown by TogetherAIEmbeddingFunction.generate() when the response from Together AI's POST https://api.together.xyz/v1/embeddings parses as JSON but has no `data` array. The client expects the OpenAI-style shape `{ data: [{ embedding: [...] }] }` and never checks `response.ok`/`response.status` first, so any JSON error body (auth error, unknown model, rate limit) lands here. Note the surrounding catch immediately re-wraps it as 'Error calling Together AI API: Invalid response format from Together AI API'.

Source

Thrown at clients/js/packages/chromadb-core/src/embeddings/TogetherAIEmbeddingFunction.ts:60

  }

  public async generate(texts: string[]): Promise<number[][]> {
    try {
      const payload = {
        model: this.model_name,
        input: texts,
      };

      const response = await fetch(ENDPOINT, {
        method: "POST",
        headers: this.headers,
        body: JSON.stringify(payload),
      });

      const resp = await response.json();

      if (!resp.data) {
        throw new Error("Invalid response format from Together AI API");
      }

      const embeddings = resp.data.map(
        (item: { embedding: number[] }) => item.embedding,
      );
      return embeddings;
    } catch (error) {
      if (error instanceof Error) {
        throw new Error(`Error calling Together AI API: ${error.message}`);
      } else {
        throw new Error(`Error calling Together AI API: ${error}`);
      }
    }
  }

  buildFromConfig(config: StoredConfig): IEmbeddingFunction {
    return new TogetherAIEmbeddingFunction({
      model_name: config.model_name,

View on GitHub (pinned to aecdd12c8a)

Solutions

1. Reproduce the exact request with curl to see the real error body: curl -s https://api.together.xyz/v1/embeddings -H "Authorization: Bearer $CHROMA_TOGETHER_AI_API_KEY" -H "Content-Type: application/json" -d '{"model":"BAAI/bge-base-en-v1.5","input":["hi"]}'
2. Fix the API key: pass together_ai_api_key in the constructor or export CHROMA_TOGETHER_AI_API_KEY.
3. Fix model_name: use a model listed under 'Embeddings' in Together's model catalog.
4. If behind a proxy/firewall, confirm it does not rewrite the response body, and test the same fetch outside Chroma.

Example fix

// before
new TogetherAIEmbeddingFunction({
  model_name: "meta-llama/Llama-3-8b-chat-hf", // not an embeddings model -> API returns error JSON without `data`
});
await collection.add({ ids: ["1"], documents: ["hello"] }); // throws

// after
new TogetherAIEmbeddingFunction({
  model_name: "BAAI/bge-base-en-v1.5", // Together embeddings model
  api_key_env_var: "CHROMA_TOGETHER_AI_API_KEY",
});
await collection.add({ ids: ["1"], documents: ["hello"] });

// Startup smoke test: fail fast with the real API error before ingesting data
const fn = new TogetherAIEmbeddingFunction({
  model_name: "BAAI/bge-base-en-v1.5",
  together_ai_api_key: process.env.CHROMA_TOGETHER_AI_API_KEY,
});
const probe = await fn.generate(["ping"]);
if (!probe?.length || !probe[0]?.length) throw new Error("Together AI embeddings misconfigured");

try {
  await collection.add({ ids, documents });
} catch (e) {
  const msg = e instanceof Error ? e.message : String(e);
  if (msg.includes("Invalid response format from Together AI API")) {
    throw new Error("Together AI rejected the embedding request (bad API key or model_name): check CHROMA_TOGETHER_AI_API_KEY and the model catalog", { cause: e });
  }
  throw e;
}

When it happens

Trigger: Any collection.add({ids, documents}) or collection.query() that makes TogetherAIEmbeddingFunction call the endpoint with: an invalid/expired API key (Bearer token rejected, body like {message:...}), a model_name that is not a Together embeddings model, a 429/5xx whose JSON body lacks `data`, or a proxy returning a JSON error envelope.

Common situations: CHROMA_TOGETHER_AI_API_KEY (or custom api_key_env_var) unset or wrong in CI/deploy env; passing a chat/completion model name (e.g. 'meta-llama/Llama-3-8b') instead of an embeddings model (e.g. 'BAAI/bge-base-en-v1.5', 'togethercomputer/m2-bert-80M-8k-retrieval'); corporate gateway intercepting the request; Together API incident.

Related errors

• resp.detail || "Unknown error"
• Failed to generate embeddings: ${response.statusText}
• data.detail
• Together AI API key is required. Please provide it in the co
• Error calling Together AI API: ${error.message}

AI-assisted analysis of chroma-core/chroma@aecdd12c8a (2026-08-16). Data as JSON: /api/errors/e4cdc65b0c4e7fb2. Report an issue: GitHub.