{"record":{"id":"83dc5f0ff7290b41","repo":"Mintplex-Labs/anything-llm","slug":"could-not-load-this-model-into-foundry-local","errorCode":null,"errorMessage":"Could not load ${this.model} into Foundry Local: ${error}","messagePattern":"Could not load (.+?) into Foundry Local: (.+?)","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"server/utils/AiProviders/foundry/index.js","lineNumber":98,"sourceCode":"  async assertModelLoaded() {\n    if (!this.model || FoundryLLM.#loadedModels.has(this.model)) return;\n    const FoundryModels = require(\"./models.js\");\n\n    // The service reports fully-qualified variant ids while the preference is\n    // usually an alias, so match on either side of the colon-versioned name.\n    const loaded = await FoundryModels.loadedModels();\n    const isLoaded = loaded.some(\n      (id) => id === this.model || id.split(\":\")[0] === this.model\n    );\n    if (isLoaded) {\n      FoundryLLM.#loadedModels.add(this.model);\n      return;\n    }\n\n    this.#log(`Loading ${this.model} into Foundry Local...`);\n    const { success, error } = await FoundryModels.loadModel(this.model);\n    if (!success)\n      throw new Error(\n        `Could not load ${this.model} into Foundry Local: ${error}`\n      );\n    FoundryLLM.#loadedModels.add(this.model);\n  }\n\n  /**\n   * Turn a mid-stream failure into something actionable.\n   *\n   * A model evicted after we loaded it — by an idle timeout, or from the host —\n   * makes the service answer 200 and then drop the socket, which reaches us\n   * only as \"Premature close\". Forget it so the next message reloads it.\n   * @param {Error} error\n   * @param {string} model\n   * @returns {string}\n   */\n  static explainStreamError(error, model) {\n    const isPrematureClose =\n      error?.code === \"ERR_STREAM_PREMATURE_CLOSE\" ||","sourceCodeStart":80,"sourceCodeEnd":116,"githubUrl":"https://github.com/Mintplex-Labs/anything-llm/blob/3aec848f2885144aa8f1e53b9731a04310d5d558/server/utils/AiProviders/foundry/index.js#L80-L116","documentation":"FoundryLLM.assertModelLoaded calls FoundryModels.loadModel(this.model); when that returns { success: false, error }, it rethrows as 'Could not load <model> into Foundry Local: <error>'. The embedded error is either the non-OK HTTP status from the service's /models/load endpoint (see error 149) or 'No Foundry service or model was set.' The class keeps a #loadedModels memo and also forgets evicted models after mid-stream failures so the next message attempts a reload — meaning this error is how load failures surface per chat message.","triggerScenarios":"getChatCompletion/streamGetChatCompletion on a model that is not currently loaded in Foundry Local, where the subsequent load call fails: the model was never downloaded (HTTP 404 from /models/load), the service is unreachable so the fetch throws, FOUNDRY_BASE_PATH is blank so origin resolution fails, or the load exceeded the multi-GB LOAD_TIMEOUT_MS and aborted.","commonSituations":"User picked a model id in AnythingLLM that exists in the catalog but was never pulled with `foundry model download`; Foundry Local service stopped or the port changed; machine low on RAM/disk so loading times out; model was loaded earlier but evicted by idle timeout and the reload fails.","solutions":["Download the model outside AnythingLLM: `foundry model download <modelId>` (or via Docker Desktop), then retry the message","Confirm the Foundry Local service is running and FOUNDRY_BASE_PATH points at it (curl the /models endpoint)","Check RAM/disk: multi-GB loads abort on the timeout when the machine is starved — free resources or pick a smaller model","Verify the model id matches exactly what `foundry model list` shows (alias vs full id)"],"exampleFix":"# before: model chosen in UI but never pulled\n# -> Could not load ai/qwen2.5-0.5b into Foundry Local: Foundry could not load ... (HTTP 404)\n\n# after\nfoundry model download ai/qwen2.5-0.5b-instruct\n# then resend the chat message","handlingStrategy":"retry","validationCode":"// pre-check that the model is downloaded/loaded before chatting\nconst loaded = await FoundryModels.loadedModels();\nif (!loaded.some((id) => id === model || id.split(\":\")[0] === model)) {\n  const { success, error } = await FoundryModels.loadModel(model);\n  if (!success) throw new Error(`Cannot preload ${model}: ${error}`);\n}","typeGuard":null,"tryCatchPattern":"try {\n  await llm.streamGetChatCompletion(messages);\n} catch (err) {\n  if (/Could not load .* into Foundry Local/.test(err.message)) {\n    // evictions/idle timeouts are transient: one reload attempt is reasonable\n    if (++attempt === 1) return retryMessage();\n    return respond(`Foundry could not load the model: ${err.message}`);\n  }\n  throw err;\n}","preventionTips":["Preload models with foundry model download before pointing workspaces at them","Keep a small set of models loaded; avoid idle-eviction churn by disabling aggressive timeouts","Monitor free RAM before requesting multi-GB loads"],"tags":["foundry-local","model-load-failed","local-inference","memory"],"backgroundTag":"model-load-failed","analyzedSha":"3aec848f2885144aa8f1e53b9731a04310d5d558","analyzedAt":"2026-08-18T10:02:21.017Z","contentChangedAt":"2026-08-18T10:02:21.017Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}