mem0ai/mem0 · error

vLLM LLM failed: ${message}

Error message

vLLM LLM failed: ${message}

What it means

Thrown by the vLLM LLM wrapper in the OSS TypeScript SDK when the OpenAI-compatible request to a self-hosted vLLM server fails during generateResponse. vLLM exposes an OpenAI-compatible /v1/chat/completions endpoint; the wrapper delegates to the parent OpenAILLM class and re-throws any failure prefixed with 'vLLM LLM failed' plus the original message.

Source

Thrown at mem0-ts/src/oss/src/llms/vllm.ts:37

    super({
      ...config,
      apiKey: config.apiKey || process.env.VLLM_API_KEY || DEFAULT_API_KEY,
      baseURL,
      model: config.model || DEFAULT_MODEL,
    });
  }

  async generateResponse(
    messages: Message[],
    responseFormat?: { type: string },
    tools?: any[],
  ): Promise<string | LLMResponse> {
    try {
      return await super.generateResponse(messages, responseFormat, tools);
    } catch (err) {
      const message = err instanceof Error ? err.message : String(err);
      throw new Error(`vLLM LLM failed: ${message}`);
    }
  }

  async generateChat(messages: Message[]): Promise<LLMResponse> {
    try {
      return await super.generateChat(messages);
    } catch (err) {
      const message = err instanceof Error ? err.message : String(err);
      throw new Error(`vLLM LLM failed: ${message}`);
    }
  }
}

View on GitHub (pinned to 001c235229)

Solutions

1. Inspect the appended message: 'fetch failed'/ECONNREFUSED means the server is down or baseURL is wrong; 404 model not found means the model name mismatch.
2. Confirm the vLLM server is up and the URL matches: curl $BASE_URL/v1/models from the same host.
3. Set config.llm.config.model to exactly the --served-model-name the server was started with.
4. If the server requires an API key, pass any non-empty apiKey in config (or remove the requirement server-side).
5. For context-length errors, shorten the input or restart vLLM with a larger --max-model-len.

Example fix

// before
const memory = new Memory({
  llm: { provider: 'vllm', config: { model: 'my-model' } }, // no baseURL -> wrong default
});

// after
const memory = new Memory({
  llm: {
    provider: 'vllm',
    config: {
      baseURL: process.env.VLLM_BASE_URL ?? 'http://localhost:8001/v1',
      apiKey: 'dummy', // vLLM ignores it unless --api-key is set
      model: 'my-model', // must match --served-model-name
    },
  },
});

async function assertVllmHealthy(baseURL: string, model: string, apiKey = 'dummy') {
  const res = await fetch(`${baseURL}/models`, {
    headers: { Authorization: `Bearer ${apiKey}` },
  });
  if (!res.ok) throw new Error(`vLLM unhealthy: ${res.status}`);
  const { data } = (await res.json()) as { data: { id: string }[] };
  if (!data.some((m) => m.id === model)) {
    throw new Error(`model '${model}' not served; available: ${data.map((m) => m.id).join(', ')}`);
  }
}

function isVllmError(err: unknown): boolean {
  return err instanceof Error && err.message.startsWith('vLLM LLM failed:');
}

try {
  await memory.add(text, opts);
} catch (err) {
  if (isVllmError(err)) {
    const inner = (err as Error).message.slice('vLLM LLM failed:'.length);
    if (/ECONNREFUSED|fetch failed/.test(inner)) {
      // server down: page/defer instead of retrying blindly
      throw new Error('vLLM server unreachable at configured baseURL');
    }
  }
  throw err;
}

When it happens

Trigger: config.llm.provider is 'vllm' and a Memory operation triggers LLM generation, but the request to the configured baseURL (e.g. http://localhost:8001/v1) fails: server not running, wrong port, no api-key configured on a server that requires one, model name not loaded in vLLM, or prompt longer than the server's max_model_len.

Common situations: vLLM server started with --served-model-name different from the model id passed in config; server bound to a different port or host than baseURL; forgetting that vLLM requires apiKey to be any non-empty string when started with an api key; context-length overflows because vLLM rejects requests exceeding max_model_len.

Related errors

• Together LLM failed: ${message}
• Unknown LLM provider: ${providerId}
• memory_id is required for update
• AWS Bedrock LLM failed: ${message}
• Azure OpenAI requires both API key and endpoint

AI-assisted analysis of mem0ai/mem0@001c235229 (2026-08-15). Data as JSON: /api/errors/de39127ed16cf041. Report an issue: GitHub.