abhigyanpatwari/GitNexus · error · Error

LLM returned empty response

Error message

LLM returned empty response

What it means

The non-streaming call returned HTTP 200, but the JSON body has no choices[0].message.content. The client expects an OpenAI-compatible chat-completions shape, so a 200 response without that path means the endpoint is not returning chat completions — wrong route, a gateway 'success' wrapping an error, or a genuinely empty completion.

Solutions

  1. Point `--base-url` at the chat-completions root (usually `https://host/v1`) — the client appends the completion path
  2. Reproduce with curl against $BASE_URL/chat/completions and inspect the JSON shape
  3. If the body wraps an error in a 200, fix the gateway to return proper status codes
  4. Retry once — some providers transiently return empty completions under load

Example fix

# before
gitnexus wiki --provider custom --base-url https://my-proxy.example.com

# after
gitnexus wiki --provider custom --base-url https://my-proxy.example.com/v1
Defensive patterns

Strategy: validation

Validate before calling

// Verify the endpoint returns an OpenAI-shaped body before the run:
const r = await fetch(`${baseUrl}/chat/completions`, { method: 'POST', headers, body: minimalPayload });
const j = await r.json();
if (!(j as any)?.choices?.[0]?.message?.content) {
  throw new Error('Endpoint is not returning OpenAI-compatible chat completions — fix --base-url');
}

Type guard

function isOpenAIChatCompletion(json: unknown): json is { choices: { message: { content: string } }[]; usage?: Record<string, number> } {
  return typeof json === 'object' && json !== null &&
    Array.isArray((json as any).choices) &&
    typeof (json as any).choices[0]?.message?.content === 'string';
}

Try / catch

try {
  await callLLM(prompt, config);
} catch (err) {
  if (err instanceof Error && err.message === 'LLM returned empty response') {
    // endpoint shape mismatch or transient empty completion: verify with curl, then retry once
  }
  throw err;
}

Prevention

When it happens

Trigger: `--base-url` points at a non-chat endpoint that still returns 200 JSON (root path, /models, a management API); a proxy that returns 200 with an error object instead of a status code; a model returning empty content (empty string is also falsy); API version mismatch producing a different schema.

Common situations: Custom/self-hosted servers that are OpenAI-ish but not compliant (LiteLLM misconfig, old vLLM); base URL accidentally including /models; gateway 200-wrapping errors; using --api-version with a provider that ignores it and returns a different envelope.

Related errors

AI-assisted analysis of abhigyanpatwari/GitNexus@52924ef12c (2026-08-20). Data as JSON: /api/errors/1dc49cd226b62c16. Report an issue: GitHub.

Appendix: source

Thrown at gitnexus/src/core/wiki/llm-client.ts:460

        `Azure content filter blocked this request. The prompt triggered content policy. Details: ${errorText.slice(0, 300)}`,
      );
    }

    // Any other non-OK response here is a terminal 4xx — resilientFetch
    // already retried 5xx/429 to exhaustion and would have thrown above.
    throw new Error(`LLM API error (${response.status}): ${errorText.slice(0, 500)}`);
  }

  // Streaming path
  if (useStream && response.body) {
    return await readSSEStream(response.body, options!.onChunk!);
  }

  // Non-streaming path
  const json = (await response.json()) as any;
  const choice = json.choices?.[0];
  if (!choice?.message?.content) {
    throw new Error('LLM returned empty response');
  }

  return {
    content: choice.message.content,
    promptTokens: json.usage?.prompt_tokens,
    completionTokens: json.usage?.completion_tokens,
  };
}

/**
 * Read an SSE stream from an OpenAI-compatible streaming response.
 */
async function readSSEStream(
  body: ReadableStream<Uint8Array>,
  onChunk: (charsReceived: number) => void,
): Promise<LLMResponse> {
  const decoder = new TextDecoder();
  const reader = body.getReader();

View on GitHub (pinned to 52924ef12c)