srbhr/Resume-Matcher · error · ValueError

Empty response from LLM

Error message

Empty response from LLM

What it means

complete_json extracts the first choice's text via _extract_choice_text and raises this ValueError when the content is empty/whitespace — i.e. the provider returned a successful response with no usable text (finish_reason may be length, content filter, or an empty message).

Source

Thrown at apps/backend/app/llm.py:1263

                attempt > 0
                and config.provider == "azure_foundry"
                and reasoning_effort in ("low", "medium", "high")
            ):
                reasoning_effort = "minimal"
            if reasoning_effort:
                kwargs["reasoning_effort"] = reasoning_effort

            # JSON-012: Fallback to prompt-only JSON mode after JSON-mode failure.
            # LiteLLM registry may report support for models that the upstream
            # aggregator (OpenRouter) cannot actually serve with response_format.
            if use_json_mode and not json_mode_failed:
                kwargs["response_format"] = {"type": "json_object"}

            response = await router.acompletion(**kwargs)
            content = _extract_choice_text(response.choices[0])

            if not content:
                raise ValueError("Empty response from LLM")

            logging.debug(
                f"LLM response (attempt {attempt + 1}): {content[:300]}")

            # Extract and parse JSON
            json_str = _extract_json(content)
            result = json.loads(json_str)

            # LLM-001: Check if parsed result appears truncated
            if isinstance(result, dict) and _appears_truncated(result, schema_type):
                if attempt < retries:
                    logging.warning(
                        "Parsed JSON appears truncated (attempt %d/%d), retrying",
                        attempt + 1,
                        retries + 1,
                    )
                    if schema_type == "resume":
                        hint = (

View on GitHub (pinned to 116f9cc3b0)

Solutions

1. Increase max_tokens — reasoning models consume budget with thinking before any visible output
2. Switch to a non-reasoning model or one whose thinking tags are stripped into content
3. complete_json already retries with backoff — check logs for repeated empty responses and the finish_reason
4. Verify the provider isn't applying content filters to your prompt

Example fix

// before
kwargs["max_tokens"] = 256;
// after
kwargs["max_tokens"] = 4096; // leave headroom for reasoning tokens

def response_has_text(resp) -> bool:
    try:
        return bool(resp["choices"][0]["message"]["content"].strip())
    except (KeyError, IndexError, AttributeError, TypeError):
        return False

function hasChoiceText(resp: unknown): resp is { choices: { message: { content: string } }[] } {
  const r = resp as any;
  return Array.isArray(r?.choices) && r.choices.length > 0 &&
    typeof r.choices[0]?.message?.content === 'string' &&
    r.choices[0].message.content.trim().length > 0;
}

try:
    data = await complete_json(prompt)
except ValueError as e:
    if e.message.includes('Empty response from LLM'):
        data = await complete_json(prompt, { maxTokens: 8192 })  // retry with more headroom
    else:
        throw e

When it happens

Trigger: router.acompletion succeeds but choices[0] has empty text — e.g. max_tokens too low so all budget went to thinking tokens, content filter stripped output, or a reasoning model emitted only hidden reasoning.

Common situations: max_tokens set below the model's thinking overhead on reasoning models (deepseek-r1/qwq); provider content-filter; transient provider bug returning an empty message.

Related errors

• LLM completion failed. Please check your API configuration a
• JSON extraction exceeded max recursion depth: {_depth}
• Content too large for JSON extraction: {len(content)} bytes
• No JSON found in response: {original[:200]}
• Failed to parse JSON after {retries + 1} attempts: {e}

AI-assisted analysis of srbhr/Resume-Matcher@116f9cc3b0 (2026-08-28). Data as JSON: /api/errors/bc944e1cc946cbe6. Report an issue: GitHub.