Mintplex-Labs/anything-llm · error · Error

LiteLLM Failed to embed

Error message

LiteLLM Failed to embed: ${error}

What it means

Thrown from LiteLLMEmbedder.embedChunks after all chunk requests settle; failures are de-duplicated into a Set and joined with commas, so the text after the colon lists each distinct error the LiteLLM proxy returned. The class pushes up to 500 strings per batch, so a misconfigured proxy fails fast and loudly.

Solutions

  1. Read each comma-separated entry — they are the LiteLLM proxy's error strings and name the exact cause
  2. For 401/403, verify LITE_LLM_API_KEY against /key/info on the proxy and confirm it can call the model
  3. For model errors, make sure EMBEDDING_MODEL_PREF matches a model_name/deployment in config.yaml and the proxy was restarted after edits
  4. For 429s, raise or wait out the key budget/tpm limits, or reduce EMBEDDING_MODEL_MAX_CHUNK_LENGTH to soften request sizes
  5. Confirm the proxy is reachable at LITE_LLM_BASE_PATH (curl the /models endpoint)
Defensive patterns

Strategy: try-catch

Validate before calling

// Pre-flight: proxy reachable and model routable before a 500-chunk batch
async function liteLLMReady(openai, model) {
  try {
    const res = await openai.embeddings.create({ model, input: ["ping"] });
    return Array.isArray(res?.data?.[0]?.embedding);
  } catch (e) {
    console.error("LiteLLM pre-flight failed:", e.status, e.message);
    return false;
  }
}

Try / catch

try {
  const vectors = await embedder.embedTextInput(text);
} catch (e) {
  if (e.message.startsWith("LiteLLM Failed to embed:")) {
    const detail = e.message.slice("LiteLLM Failed to embed:".length);
    if (/401|403|auth/i.test(detail)) { /* fix LITE_LLM_API_KEY / virtual key budgets; no retry */ }
    else if (/429|rate|budget/i.test(detail)) { /* wait out budget window or raise limits, then retry */ }
    else if (/404|model/i.test(detail)) { /* align EMBEDDING_MODEL_PREF with config.yaml, restart proxy, retry */ }
    else throw e;
  } else throw e;
}

Prevention

When it happens

Trigger: 401/403 because LITE_LLM_API_KEY is not a valid virtual key or lacks the embedding model; 404/BadRequest because EMBEDDING_MODEL_PREF is not a model or deployment name in the proxy's config.yaml; 429 because the key's budget, tpm/rpm limits, or upstream provider quota is exceeded; upstream provider auth failure (e.g. proxy's OPENAI_API_KEY invalid); proxy not running at LITE_LLM_BASE_PATH.

Common situations: LiteLLM gateway whose config.yaml was changed but the proxy not restarted; virtual keys with tight budgets during bulk re-embedding; routing to an Azure/OpenAI deployment whose deployed name differs from the model name; proxy listening on a different port than configured.

Related errors

AI-assisted analysis of Mintplex-Labs/anything-llm@3aec848f28 (2026-08-18). Data as JSON: /api/errors/11eb380aa1d2ee31. Report an issue: GitHub.

Appendix: source

Thrown at server/utils/EmbeddingEngines/liteLLM/index.js:92

        .flat();
      if (errors.length > 0) {
        let uniqueErrors = new Set();
        errors.map((error) =>
          uniqueErrors.add(`[${error.type}]: ${error.message}`)
        );

        return {
          data: [],
          error: Array.from(uniqueErrors).join(", "),
        };
      }
      return {
        data: results.map((res) => res?.data || []).flat(),
        error: null,
      };
    });

    if (!!error) throw new Error(`LiteLLM Failed to embed: ${error}`);
    return data.length > 0 &&
      data.every((embd) => embd.hasOwnProperty("embedding"))
      ? data.map((embd) => embd.embedding)
      : null;
  }
}

module.exports = {
  LiteLLMEmbedder,
};

View on GitHub (pinned to 3aec848f28)