What causes "LocalAI Failed to embed: ${error}" in Mintplex-Labs/anything-llm?

Read the joined message — each entry is LocalAI's own error string (model not found, backend load failure, auth, etc.)

Mintplex-Labs/anything-llm · error · Error

LocalAI Failed to embed

Error message

LocalAI Failed to embed: ${error}

What it means

Thrown from LocalAiEmbedder.embedChunks when any chunk request to LocalAI fails; distinct errors are collected into a Set and joined with commas, so the text after the colon enumerates each unique upstream failure. Requests are batched up to 50 chunks at once, and a single bad batch aborts the embedding.

Solutions

Read the joined message — each entry is LocalAI's own error string (model not found, backend load failure, auth, etc.)
Verify the model is installed and responds: curl the /v1/embeddings endpoint directly with a one-line input
Reinstall or pull the embedding model in the LocalAI gallery and align EMBEDDING_MODEL_PREF with its exact name
Set LOCAL_AI_API_KEY if your LocalAI instance enforces auth
For memory/context failures, lower EMBEDDING_MODEL_MAX_CHUNK_LENGTH or reduce concurrency/batch load

Example fix

# verify the model actually embeds before blaming AnythingLLM:
curl http://localhost:8080/v1/embeddings \
  -H 'Content-Type: application/json' \
  -d '{"model":"bert-embeddings","input":["hello"]}'
# 200 -> fix EMBEDDING_MODEL_PREF to match; 404 -> install the model in LocalAI

Defensive patterns

Strategy: try-catch

Validate before calling

// Smoke-test LocalAI exactly like production will: POST /embeddings with one input
async function localAiEmbedderHealthy(basePath, model, apiKey) {
  const res = await fetch(`${basePath}/embeddings`, {
    method: "POST",
    headers: { "Content-Type": "application/json", ...(apiKey ? { Authorization: `Bearer ${apiKey}` } : {}) },
    body: JSON.stringify({ model, input: ["ping"] }),
  });
  if (!res.ok) { console.error("LocalAI pre-flight:", res.status, await res.text()); return false; }
  const json = await res.json();
  return Array.isArray(json?.data?.[0]?.embedding);
}

Try / catch

try {
  const vectors = await embedder.embedTextInput(text);
} catch (e) {
  if (e.message.startsWith("LocalAI Failed to embed:")) {
    const detail = e.message;
    if (/not found|404/i.test(detail)) { /* install the model in the LocalAI gallery; no retry until fixed */ }
    else if (/401|unauthorized/i.test(detail)) { /* set LOCAL_AI_API_KEY */ }
    else if (/429|load|memory/i.test(detail)) { /* reduce batch load / free resources, then retry once */ }
    else throw e;
  } else throw e;
}

Prevention

Verify the model responds to a single-input /embeddings call before bulk embedding
Watch LocalAI logs during embedding — its backend load errors are the root cause of many normalized messages
Size EMBEDDING_MODEL_MAX_CHUNK_LENGTH to the model's limit to avoid rejected batches of 50 chunks

When it happens

Trigger: EMBEDDING_MODEL_PREF names a model that is not installed in LocalAI (404 error: 'model not found'); the model exists but its backend (e.g. sentence-transformers) is missing or fails to load; 401 when LocalAI requires LOCAL_AI_API_KEY and it is unset/wrong; chunk sizes exceeding the model's max length; LocalAI restarting/crashing under the 50-chunk batch load.

Common situations: Gallery model half-installed after an interrupted pull; LocalAI upgraded and backend names changed; CPU/RAM exhaustion on small hosts during bulk embedding causing worker errors.

Related errors

AI-assisted analysis of Mintplex-Labs/anything-llm@3aec848f28 (2026-08-18). Data as JSON: /api/errors/9242995112a7d104. Report an issue: GitHub.

Appendix: source

Thrown at server/utils/EmbeddingEngines/localAi/index.js:116

        .flat();
      if (errors.length > 0) {
        let uniqueErrors = new Set();
        errors.map((error) =>
          uniqueErrors.add(`[${error.type}]: ${error.message}`)
        );

        return {
          data: [],
          error: Array.from(uniqueErrors).join(", "),
        };
      }
      return {
        data: results.map((res) => res?.data || []).flat(),
        error: null,
      };
    });

    if (!!error) throw new Error(`LocalAI Failed to embed: ${error}`);
    return data.length > 0 &&
      data.every((embd) => embd.hasOwnProperty("embedding"))
      ? data.map((embd) => embd.embedding)
      : null;
  }
}

module.exports = {
  LocalAiEmbedder,
};

View on GitHub (pinned to 3aec848f28)