Mintplex-Labs/anything-llm · error · Error

Foundry could not load ${modelId} (HTTP ${response.status}).

Error message

Foundry could not load ${modelId} (HTTP ${response.status}). Is the model downloaded?

What it means

FoundryModels.loadModel issues GET <origin>/models/load/<modelId> with a long timeout (loading pulls multi-GB into memory) and throws this error when the service responds with a non-OK status. It is returned as { success: false, error } to the caller (assertModelLoaded, error 145), not thrown to user code directly. The 'Is the model downloaded?' hint reflects the most common cause: the catalog lists the model but bits were never pulled to disk.

Source

Thrown at server/utils/AiProviders/foundry/models.js:141

   * so this has to happen before the first completion — otherwise a streaming
   * request is answered with headers and then the connection is dropped.
   * @param {string} modelId - Alias or fully-qualified variant id.
   * @param {string} basePath
   * @returns {Promise<{success: boolean, error: string|null}>}
   */
  static async loadModel(modelId, basePath = process.env.FOUNDRY_BASE_PATH) {
    const origin = this.#originOf(basePath);
    if (!origin || !modelId)
      return { success: false, error: "No Foundry service or model was set." };

    try {
      // Loading pulls a multi-GB model into memory, well past the probe timeout.
      const response = await fetch(
        `${origin}/models/load/${encodeURIComponent(modelId)}`,
        { signal: AbortSignal.timeout(this.LOAD_TIMEOUT_MS) }
      );
      if (!response.ok)
        throw new Error(
          `Foundry could not load ${modelId} (HTTP ${response.status}). Is the model downloaded?`
        );
      return { success: true, error: null };
    } catch (e) {
      return { success: false, error: e.message };
    }
  }

  /**
   * Release a model from memory.
   * @param {string} modelId
   * @param {string} basePath
   * @returns {Promise<boolean>}
   */
  static async unloadModel(modelId, basePath = process.env.FOUNDRY_BASE_PATH) {
    const origin = this.#originOf(basePath);
    if (!origin || !modelId) return false;
    try {

View on GitHub (pinned to 3aec848f28)

Solutions

1. Download the model first: `foundry model download <modelId>` and confirm with `foundry model list`
2. Use the exact id the service reports (full 'ai/...' id, not a display alias)
3. Free memory or choose a smaller quantization; check Foundry Local logs for the 5xx reason
4. Update Foundry Local so its /models/load endpoint and catalog are current

Example fix

# before
# -> Foundry could not load ai/phi-3.5-mini (HTTP 404). Is the model downloaded?

# after
foundry model download ai/phi-3.5-mini-instruct
# retry the chat message

// verify the model exists on the service before asking it to load
const res = await fetch(`${origin}/models`);
const { models } = await res.json();
if (!models.some((m) => m.id === modelId)) {
  throw new Error(`Model ${modelId} not present on Foundry — download it first.`);
}
const { success, error } = await FoundryModels.loadModel(modelId);

const { success, error } = await FoundryModels.loadModel(modelId);
if (!success) {
  if (/HTTP 404/.test(error)) return guideDownload(modelId); // permanent: needs `foundry model download`
  if (/HTTP 5/.test(error)) return retryAfterDelay();        // service-side, may clear
  throw new Error(`Foundry load failed: ${error}`);
}

When it happens

Trigger: GET /models/load/<id> returning 404 (model not downloaded / unknown id), 400 (bad id form), or a 5xx when the Foundry service fails to allocate memory. Also triggered when the fetch itself aborts past LOAD_TIMEOUT_MS (then the catch returns the abort message instead), or when basePath/modelId are blank (the 'No Foundry service or model was set.' branch above).

Common situations: Model id copied from the catalog without running `foundry model download`; Foundry Local running an old version without that model; insufficient RAM so the load endpoint errors; modelId alias vs full-id mismatch (the caller also matches id.split(':')[0]).

Related errors

• Could not load ${this.model} into Foundry Local: ${error}
• No Foundry Base Path was set.
• Foundry chat: ${this.model} is not valid or defined model fo
• e.message
• e.message

AI-assisted analysis of Mintplex-Labs/anything-llm@3aec848f28 (2026-08-18). Data as JSON: /api/errors/a7fdf5100cb84a18. Report an issue: GitHub.