{"record":{"id":"de39127ed16cf041","repo":"mem0ai/mem0","slug":"vllm-llm-failed-message","errorCode":null,"errorMessage":"vLLM LLM failed: ${message}","messagePattern":"vLLM LLM failed: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"mem0-ts/src/oss/src/llms/vllm.ts","lineNumber":37,"sourceCode":"\n    super({\n      ...config,\n      apiKey: config.apiKey || process.env.VLLM_API_KEY || DEFAULT_API_KEY,\n      baseURL,\n      model: config.model || DEFAULT_MODEL,\n    });\n  }\n\n  async generateResponse(\n    messages: Message[],\n    responseFormat?: { type: string },\n    tools?: any[],\n  ): Promise<string | LLMResponse> {\n    try {\n      return await super.generateResponse(messages, responseFormat, tools);\n    } catch (err) {\n      const message = err instanceof Error ? err.message : String(err);\n      throw new Error(`vLLM LLM failed: ${message}`);\n    }\n  }\n\n  async generateChat(messages: Message[]): Promise<LLMResponse> {\n    try {\n      return await super.generateChat(messages);\n    } catch (err) {\n      const message = err instanceof Error ? err.message : String(err);\n      throw new Error(`vLLM LLM failed: ${message}`);\n    }\n  }\n}\n","sourceCodeStart":19,"sourceCodeEnd":50,"githubUrl":"https://github.com/mem0ai/mem0/blob/001c235229be8795e3834520467bd0d661ed8f34/mem0-ts/src/oss/src/llms/vllm.ts#L19-L50","documentation":"Thrown by the vLLM LLM wrapper in the OSS TypeScript SDK when the OpenAI-compatible request to a self-hosted vLLM server fails during generateResponse. vLLM exposes an OpenAI-compatible /v1/chat/completions endpoint; the wrapper delegates to the parent OpenAILLM class and re-throws any failure prefixed with 'vLLM LLM failed' plus the original message.","triggerScenarios":"config.llm.provider is 'vllm' and a Memory operation triggers LLM generation, but the request to the configured baseURL (e.g. http://localhost:8001/v1) fails: server not running, wrong port, no api-key configured on a server that requires one, model name not loaded in vLLM, or prompt longer than the server's max_model_len.","commonSituations":"vLLM server started with --served-model-name different from the model id passed in config; server bound to a different port or host than baseURL; forgetting that vLLM requires apiKey to be any non-empty string when started with an api key; context-length overflows because vLLM rejects requests exceeding max_model_len.","solutions":["Inspect the appended message: 'fetch failed'/ECONNREFUSED means the server is down or baseURL is wrong; 404 model not found means the model name mismatch.","Confirm the vLLM server is up and the URL matches: curl $BASE_URL/v1/models from the same host.","Set config.llm.config.model to exactly the --served-model-name the server was started with.","If the server requires an API key, pass any non-empty apiKey in config (or remove the requirement server-side).","For context-length errors, shorten the input or restart vLLM with a larger --max-model-len."],"exampleFix":"// before\nconst memory = new Memory({\n  llm: { provider: 'vllm', config: { model: 'my-model' } }, // no baseURL -> wrong default\n});\n\n// after\nconst memory = new Memory({\n  llm: {\n    provider: 'vllm',\n    config: {\n      baseURL: process.env.VLLM_BASE_URL ?? 'http://localhost:8001/v1',\n      apiKey: 'dummy', // vLLM ignores it unless --api-key is set\n      model: 'my-model', // must match --served-model-name\n    },\n  },\n});","handlingStrategy":"try-catch","validationCode":"async function assertVllmHealthy(baseURL: string, model: string, apiKey = 'dummy') {\n  const res = await fetch(`${baseURL}/models`, {\n    headers: { Authorization: `Bearer ${apiKey}` },\n  });\n  if (!res.ok) throw new Error(`vLLM unhealthy: ${res.status}`);\n  const { data } = (await res.json()) as { data: { id: string }[] };\n  if (!data.some((m) => m.id === model)) {\n    throw new Error(`model '${model}' not served; available: ${data.map((m) => m.id).join(', ')}`);\n  }\n}","typeGuard":"function isVllmError(err: unknown): boolean {\n  return err instanceof Error && err.message.startsWith('vLLM LLM failed:');\n}","tryCatchPattern":"try {\n  await memory.add(text, opts);\n} catch (err) {\n  if (isVllmError(err)) {\n    const inner = (err as Error).message.slice('vLLM LLM failed:'.length);\n    if (/ECONNREFUSED|fetch failed/.test(inner)) {\n      // server down: page/defer instead of retrying blindly\n      throw new Error('vLLM server unreachable at configured baseURL');\n    }\n  }\n  throw err;\n}","preventionTips":["Run a startup health check against /v1/models and fail deployment if it errors.","Keep --served-model-name and config.llm.config.model in one source of truth (env/CI var).","Always pass a non-empty apiKey in vllm config — the OpenAI client refuses empty keys."],"tags":["llm","vllm","self-hosted","typescript","oss-sdk"],"backgroundTag":null,"analyzedSha":"001c235229be8795e3834520467bd0d661ed8f34","analyzedAt":"2026-08-15T01:55:42.685Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}