Mintplex-Labs/anything-llm · error · Error

LiteLLM Failed to embed: ${error}

Error message

LiteLLM Failed to embed: ${error}

What it means

Thrown at the end of embedChunks after Promise.all when at least one concurrent batch request to the LiteLLM endpoint rejected. Each batch promise is caught and resolved with {data:[],error:e}, errors are deduplicated into a [type]: message string, and if any exist the whole sequence is aborted because partial embeddings would corrupt the vector store. The error.type is derived from response.data.error.code, the HTTP status, or defaults to 'failed_to_embed'.

Source

Thrown at server/utils/EmbeddingEngines/liteLLM/index.js:92

        .flat();
      if (errors.length > 0) {
        let uniqueErrors = new Set();
        errors.map((error) =>
          uniqueErrors.add(`[${error.type}]: ${error.message}`)
        );

        return {
          data: [],
          error: Array.from(uniqueErrors).join(", "),
        };
      }
      return {
        data: results.map((res) => res?.data || []).flat(),
        error: null,
      };
    });

    if (!!error) throw new Error(`LiteLLM Failed to embed: ${error}`);
    return data.length > 0 &&
      data.every((embd) => embd.hasOwnProperty("embedding"))
      ? data.map((embd) => embd.embedding)
      : null;
  }
}

module.exports = {
  LiteLLMEmbedder,
};

View on GitHub (pinned to 526360e320)

Solutions

1. Read the bracketed [type] and message in the thrown string: a 401 means fix LITE_LLM_API_KEY, a 4xx model error means fix EMBEDDING_MODEL_PREF to match a LiteLLM-deployed alias
2. curl the LiteLLM proxy /v1/embeddings directly with the same model and key to isolate proxy vs provider failure
3. If batches are too large (near the ~8MB POST ceiling), lower the workload chunk size rather than maxConcurrentChunks
4. Restart the LiteLLM proxy and retry; check its logs for the upstream error

Example fix

// before
EMBEDDING_MODEL_PREF=text-embedding-ada-002   // not deployed in LiteLLM

// after
EMBEDDING_MODEL_PREF=azure/ada-002            // alias configured in LiteLLM config.yaml

// sanity-check the model is deployed before a large ingest
async function litellmModelExists(openai, model) {
  const res = await openai.models.list();
  return res.data.some(m => m.id === model);
}

function isEmbeddingError(e) {
  return e instanceof Error && /LiteLLM Failed to embed/.test(e.message);
}

try {
  return await embedder.embedChunks(chunks);
} catch (e) {
  if (/401|failed_to_embed/.test(e.message)) throw e;          // not transient
  // 429/5xx are often transient — one bounded retry with backoff
  await new Promise(r => setTimeout(r, 1000));
  return await embedder.embedChunks(chunks);
}

When it happens

Trigger: Any of the maxConcurrentChunks (500) batched POST /embeddings calls failing: HTTP 4xx from a wrong model name (EMBEDDING_MODEL_PREF not deployed in LiteLLM), 401 auth (missing/wrong LITE_LLM_API_KEY upstream), 5xx from the proxied provider, network timeout, or response payloads missing response.data.error.message so e.message is undefined.

Common situations: LiteLLM proxy model alias mistyped; upstream provider outage or rate limit; LiteLLM proxy restarted/under load so the 8MB POST limit is exceeded by a large batch; API key expired on the proxied provider; reverse proxy/gateway returning HTML error pages that break the openai SDK JSON parse.

Related errors

• LocalAI Failed to embed: ${error}
• LMStudio service could not be reached. Is LMStudio running?
• LMStudio Failed to embed: ${Array.from(uniqueErrors).join(",
• Ollama service could not be reached. Is Ollama running?
• Ollama Failed to embed: ${error}

AI-assisted analysis of Mintplex-Labs/anything-llm@526360e320 (2026-08-13). Data as JSON: /api/errors/11eb380aa1d2ee31. Report an issue: GitHub.