Mintplex-Labs/anything-llm · error · Error

Could not load ${this.model} into Foundry Local: ${error}

Error message

Could not load ${this.model} into Foundry Local: ${error}

What it means

Thrown by assertModelLoaded after FoundryModels.loadModel(this.model) returns { success: false }. AnythingLLM first lists already-loaded models and caches them; if the requested model is neither loaded nor loadable into Foundry Local it surfaces Foundry's own failure reason (the `error` field). The model string must match a catalog entry Foundry Local recognizes, including any precision/tag suffix handling done by the id === this.model or id.split(':')[0] comparison.

Source

Thrown at server/utils/AiProviders/foundry/index.js:98

  async assertModelLoaded() {
    if (!this.model || FoundryLLM.#loadedModels.has(this.model)) return;
    const FoundryModels = require("./models.js");

    // The service reports fully-qualified variant ids while the preference is
    // usually an alias, so match on either side of the colon-versioned name.
    const loaded = await FoundryModels.loadedModels();
    const isLoaded = loaded.some(
      (id) => id === this.model || id.split(":")[0] === this.model
    );
    if (isLoaded) {
      FoundryLLM.#loadedModels.add(this.model);
      return;
    }

    this.#log(`Loading ${this.model} into Foundry Local...`);
    const { success, error } = await FoundryModels.loadModel(this.model);
    if (!success)
      throw new Error(
        `Could not load ${this.model} into Foundry Local: ${error}`
      );
    FoundryLLM.#loadedModels.add(this.model);
  }

  /**
   * Turn a mid-stream failure into something actionable.
   *
   * A model evicted after we loaded it — by an idle timeout, or from the host —
   * makes the service answer 200 and then drop the socket, which reaches us
   * only as "Premature close". Forget it so the next message reloads it.
   * @param {Error} error
   * @param {string} model
   * @returns {string}
   */
  static explainStreamError(error, model) {
    const isPrematureClose =
      error?.code === "ERR_STREAM_PREMATURE_CLOSE" ||

View on GitHub (pinned to 526360e320)

Solutions

1. Check the exact model id with `foundry model list` and set FOUNDRY_MODEL_PREF (or pass modelPreference) to a name that appears there verbatim
2. Pre-load the model manually with `foundry model load <id>` and read the error it prints, then retry
3. Free disk space and verify GPU/VRAM headroom, then retry the load
4. Update Foundry Local so its catalog knows the requested model id

Example fix

// before
//   FOUNDRY_MODEL_PREF=phi-3-mini   // Foundry expects 'Phi-3-mini-4k-instruct-cuda'

// after
//   FOUNDRY_MODEL_PREF=Phi-3-mini-4k-instruct-cuda

// verify the model id is in Foundry's catalog before first use
const loaded = await FoundryModels.loadedModels();
const known = loaded.some(id => id === model || id.split(':')[0] === model);
if (!known) {
  // optionally prompt the user to pre-load it rather than failing mid-chat
}

try {
  await llm.getChatCompletion(messages);
} catch (e) {
  if (/Could not load .* into Foundry Local/.test(e.message)) {
    // the model is unavailable — surface a model-picker UI, don't auto-retry blindly
  }
  throw e;
}

When it happens

Trigger: Calling any chat/stream path that triggers assertModelLoaded() (getChatCompletion, streamGetChatCompletion) for a model that Foundry Local cannot resolve or download — wrong model id, a model not present in the local catalog, insufficient disk/GPU to pull it, or a transient Foundry service error during load.

Common situations: Typo or stale model name in FOUNDRY_MODEL_PREF (e.g. missing the vendor prefix Foundry expects); requesting a model that needs a larger GPU than available; Foundry Local catalog out of date; offline machine where the model was never pre-downloaded; quant/precision suffix mismatch so the id.split(':')[0] check never matches.

Related errors

• No Foundry Base Path was set.
• Foundry chat: ${this.model} is not valid or defined model fo
• KoboldCPP must have a valid base path to use for the api.
• Foundry Local crashed trying to reply to this message. You s
• error.message

AI-assisted analysis of Mintplex-Labs/anything-llm@526360e320 (2026-08-13). Data as JSON: /api/errors/83dc5f0ff7290b41. Report an issue: GitHub.