Mintplex-Labs/anything-llm · error · Error

LocalAI Failed to embed: ${error}

Error message

LocalAI Failed to embed: ${error}

What it means

Thrown at the end of embedChunks after Promise.all when any concurrent batch to the LocalAI endpoint failed. Batches of maxConcurrentChunks (50) are sent via embeddings.create; each rejection is caught and resolved as {data:[],error:e}, errors are deduplicated to a joined string, and presence of any error aborts the sequence to avoid incomplete vector data. This mirrors the LiteLLM/OpenAI batch flow.

Source

Thrown at server/utils/EmbeddingEngines/localAi/index.js:116

        .flat();
      if (errors.length > 0) {
        let uniqueErrors = new Set();
        errors.map((error) =>
          uniqueErrors.add(`[${error.type}]: ${error.message}`)
        );

        return {
          data: [],
          error: Array.from(uniqueErrors).join(", "),
        };
      }
      return {
        data: results.map((res) => res?.data || []).flat(),
        error: null,
      };
    });

    if (!!error) throw new Error(`LocalAI Failed to embed: ${error}`);
    return data.length > 0 &&
      data.every((embd) => embd.hasOwnProperty("embedding"))
      ? data.map((embd) => embd.embedding)
      : null;
  }
}

module.exports = {
  LocalAiEmbedder,
};

View on GitHub (pinned to 526360e320)

Solutions

1. Decode the joined [type]: message — a 401/403 means set LOCAL_AI_API_KEY, a 404 means fix EMBEDDING_MODEL_PREF to a loaded model
2. curl the LocalAI /v1/embeddings endpoint with the same model, key, and a sample input to reproduce
3. Reduce document chunk size or lower batch volume if LocalAI is OOMing
4. Check LocalAI logs for the per-request upstream error

Example fix

// before
// LOCAL_AI_API_KEY unset but LocalAI requires auth -> 401 in joined error

// after
EMBEDDING_BASE_PATH=http://localhost:8080/v1
EMBEDDING_MODEL_PREF=bge-small-en
LOCAL_AI_API_KEY=my-local-key

// verify auth + model before the bulk run
async function localAiReady(openai, model) {
  const res = await openai.models.list();
  return res.data.some(m => m.id === model);
}

function isLocalAIEmbedError(e) {
  return e instanceof Error && /LocalAI Failed to embed/.test(e.message);
}

try {
  return await embedder.embedChunks(chunks);
} catch (e) {
  if (/401|403|404/.test(e.message)) throw e;     // config error, not transient
  await new Promise(r => setTimeout(r, 1000));
  return await embedder.embedChunks(chunks);       // retry transient 5xx
}

When it happens

Trigger: One or more batched /embeddings calls rejecting: 404 model not loaded in LocalAI; 401 from a missing/wrong LOCAL_AI_API_KEY; 500 from LocalAI failing to run the model (OOM, GGUF mismatch); input too large; LocalAI returning a non-JSON error breaking SDK parsing.

Common situations: LOCAL_AI_API_KEY required by the LocalAI server but unset; model file corrupted or wrong architecture; LocalAI server under-resourced (CPU/GPU) so larger batches OOM; LocalAI version change altering the model id or response shape; network instability to a remote LocalAI instance.

Related errors

• LiteLLM Failed to embed: ${error}
• LMStudio service could not be reached. Is LMStudio running?
• LMStudio Failed to embed: ${Array.from(uniqueErrors).join(",
• Ollama service could not be reached. Is Ollama running?
• Ollama Failed to embed: ${error}

AI-assisted analysis of Mintplex-Labs/anything-llm@526360e320 (2026-08-13). Data as JSON: /api/errors/9242995112a7d104. Report an issue: GitHub.