{"record":{"id":"83dc5f0ff7290b41","repo":"Mintplex-Labs/anything-llm","slug":"could-not-load-this-model-into-foundry-local","errorCode":null,"errorMessage":"Could not load ${this.model} into Foundry Local: ${error}","messagePattern":"Could not load (.+?) into Foundry Local: (.+?)","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"server/utils/AiProviders/foundry/index.js","lineNumber":98,"sourceCode":"  async assertModelLoaded() {\n    if (!this.model || FoundryLLM.#loadedModels.has(this.model)) return;\n    const FoundryModels = require(\"./models.js\");\n\n    // The service reports fully-qualified variant ids while the preference is\n    // usually an alias, so match on either side of the colon-versioned name.\n    const loaded = await FoundryModels.loadedModels();\n    const isLoaded = loaded.some(\n      (id) => id === this.model || id.split(\":\")[0] === this.model\n    );\n    if (isLoaded) {\n      FoundryLLM.#loadedModels.add(this.model);\n      return;\n    }\n\n    this.#log(`Loading ${this.model} into Foundry Local...`);\n    const { success, error } = await FoundryModels.loadModel(this.model);\n    if (!success)\n      throw new Error(\n        `Could not load ${this.model} into Foundry Local: ${error}`\n      );\n    FoundryLLM.#loadedModels.add(this.model);\n  }\n\n  /**\n   * Turn a mid-stream failure into something actionable.\n   *\n   * A model evicted after we loaded it — by an idle timeout, or from the host —\n   * makes the service answer 200 and then drop the socket, which reaches us\n   * only as \"Premature close\". Forget it so the next message reloads it.\n   * @param {Error} error\n   * @param {string} model\n   * @returns {string}\n   */\n  static explainStreamError(error, model) {\n    const isPrematureClose =\n      error?.code === \"ERR_STREAM_PREMATURE_CLOSE\" ||","sourceCodeStart":80,"sourceCodeEnd":116,"githubUrl":"https://github.com/Mintplex-Labs/anything-llm/blob/526360e320da9d1b36074be5ed64fe76e5bbfbbd/server/utils/AiProviders/foundry/index.js#L80-L116","documentation":"Thrown by assertModelLoaded after FoundryModels.loadModel(this.model) returns { success: false }. AnythingLLM first lists already-loaded models and caches them; if the requested model is neither loaded nor loadable into Foundry Local it surfaces Foundry's own failure reason (the `error` field). The model string must match a catalog entry Foundry Local recognizes, including any precision/tag suffix handling done by the id === this.model or id.split(':')[0] comparison.","triggerScenarios":"Calling any chat/stream path that triggers assertModelLoaded() (getChatCompletion, streamGetChatCompletion) for a model that Foundry Local cannot resolve or download — wrong model id, a model not present in the local catalog, insufficient disk/GPU to pull it, or a transient Foundry service error during load.","commonSituations":"Typo or stale model name in FOUNDRY_MODEL_PREF (e.g. missing the vendor prefix Foundry expects); requesting a model that needs a larger GPU than available; Foundry Local catalog out of date; offline machine where the model was never pre-downloaded; quant/precision suffix mismatch so the id.split(':')[0] check never matches.","solutions":["Check the exact model id with `foundry model list` and set FOUNDRY_MODEL_PREF (or pass modelPreference) to a name that appears there verbatim","Pre-load the model manually with `foundry model load <id>` and read the error it prints, then retry","Free disk space and verify GPU/VRAM headroom, then retry the load","Update Foundry Local so its catalog knows the requested model id"],"exampleFix":"// before\n//   FOUNDRY_MODEL_PREF=phi-3-mini   // Foundry expects 'Phi-3-mini-4k-instruct-cuda'\n\n// after\n//   FOUNDRY_MODEL_PREF=Phi-3-mini-4k-instruct-cuda","handlingStrategy":"retry","validationCode":"// verify the model id is in Foundry's catalog before first use\nconst loaded = await FoundryModels.loadedModels();\nconst known = loaded.some(id => id === model || id.split(':')[0] === model);\nif (!known) {\n  // optionally prompt the user to pre-load it rather than failing mid-chat\n}","typeGuard":null,"tryCatchPattern":"try {\n  await llm.getChatCompletion(messages);\n} catch (e) {\n  if (/Could not load .* into Foundry Local/.test(e.message)) {\n    // the model is unavailable — surface a model-picker UI, don't auto-retry blindly\n  }\n  throw e;\n}","preventionTips":["Pre-load and cache the model id with `foundry model load` during provisioning","Treat model-load failure as permanent until config changes — do not blind-retry","Show the user the exact Foundry catalog id to avoid suffix/prefix mismatches"],"tags":["foundry","model-loading","local-inference","configuration"],"backgroundTag":null,"analyzedSha":"526360e320da9d1b36074be5ed64fe76e5bbfbbd","analyzedAt":"2026-08-13T01:45:47.170Z","schemaVersion":2},"datasetVersion":"2026-08-13T04:17:16.726Z"}