{"record":{"id":"dc93bb1b49459048","repo":"Mintplex-Labs/anything-llm","slug":"foundry-local-crashed-trying-to-reply-to-this-mess","errorCode":null,"errorMessage":"Foundry Local crashed trying to reply to this message. You should change the message or try again.","messagePattern":"Foundry Local crashed trying to reply to this message\\. You should change the message or try again\\.","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"server/utils/agents/aibitat/providers/foundry.js","lineNumber":136,"sourceCode":"\n  async #handleFunctionCallChat({ messages = [] }) {\n    await this.#assertContextLimits();\n    return await this.client.chat.completions\n      .create({\n        model: this.model,\n        messages,\n        max_completion_tokens: FoundryLLM.promptWindowLimit(this.model),\n      })\n      .then((result) => {\n        if (!result.hasOwnProperty(\"choices\"))\n          throw new Error(\"Microsoft Foundry Local chat: No results!\");\n        if (result.choices.length === 0)\n          throw new Error(\"Microsoft Foundry Local chat: No results length!\");\n        return result.choices[0].message.content;\n      })\n      .catch((e) => {\n        if (this.#isPrematureClose(e))\n          throw new Error(FoundryLLM.explainStreamError(e, this.model));\n        return null;\n      });\n  }\n\n  async #handleFunctionCallStream({ messages = [] }) {\n    await this.#assertContextLimits();\n    return await this.client.chat.completions.create({\n      model: this.model,\n      stream: true,\n      messages,\n      max_completion_tokens: FoundryLLM.promptWindowLimit(this.model),\n    });\n  }\n\n  /**\n   * Stream a chat completion with tool calling support.\n   * Uses native tool calling when supported, otherwise falls back to UnTooled.\n   */","sourceCodeStart":118,"sourceCodeEnd":154,"githubUrl":"https://github.com/Mintplex-Labs/anything-llm/blob/526360e320da9d1b36074be5ed64fe76e5bbfbbd/server/utils/agents/aibitat/providers/foundry.js#L118-L154","documentation":"In `#handleFunctionCallChat` (the UnTooled non-streaming callback), when the OpenAI SDK throws and `#isPrematureClose(error)` is true (error.code === 'ERR_STREAM_PREMATURE_CLOSE' or message matches /premature close/i), the catch calls `FoundryLLM.explainStreamError(e, model)`. That static method deletes the model from the in-process loaded-models cache (Foundry Local answers 200 then drops the socket when its model has crashed/unloaded, so the next call must reload it) and returns the literal 'Foundry Local crashed trying to reply to this message. You should change the message or try again.' Any non-premature error in this path returns `null`, surfacing elsewhere as an empty result.","triggerScenarios":"Microsoft Foundry Local crashed the model process mid-reply; the model OOM'd on the prompt; the Foundry Local service restarted; a GPU/driver fault aborted local inference; the model was unloaded by a concurrent request.","commonSituations":"Large/long prompts causing local model OOM; Foundry Local beta instability; concurrent requests to a single-model local instance; GPU driver/firmware issue; Foundry Local upgraded and the model needs re-download.","solutions":["Retry the turn — explainStreamError already evicted the model from cache so the next call reloads it.","Shorten or simplify the prompt to avoid local OOM.","Confirm the Foundry Local service is healthy (`foundry` CLI / Windows service status).","Reduce concurrency against the local Foundry instance to one in-flight request.","If it recurs, re-pull/reload the model with the Foundry CLI and check GPU/driver health."],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// Preflight: confirm Foundry Local has the model loaded before the chat callback.\nasync function assertFoundryModelLoaded(model) {\n  await new FoundryLLM(null, model).assertModelLoaded();\n}","typeGuard":"// Detect the premature-close shape that triggers this branch.\nfunction isFoundryPrematureClose(e) {\n  return e?.code === 'ERR_STREAM_PREMATURE_CLOSE' || /premature close/i.test(e?.message ?? '');\n}","tryCatchPattern":"// The model is auto-evicted on premature close — retry once; it will reload.\ntry { return await provider.complete(messages, functions); }\ncatch (e) {\n  if (isFoundryPrematureClose(e)) return await retryAfterShortBackoff();\n  throw e;\n}","preventionTips":["Keep prompts within local GPU memory to avoid OOM-induced crashes.","Limit concurrency to one in-flight Foundry request.","Confirm Foundry Local service health before relying on it.","After a premature close, the next call auto-reloads — do not also manually reload."],"tags":["llm-provider","foundry","local-inference","premature-close","stream-error"],"backgroundTag":null,"analyzedSha":"526360e320da9d1b36074be5ed64fe76e5bbfbbd","analyzedAt":"2026-08-13T01:45:47.170Z","schemaVersion":2},"datasetVersion":"2026-08-13T04:17:16.726Z"}