HKUDS/DeepTutor · error · ValueError

API returned no choices in response

Error message

API returned no choices in response

What it means

The OpenAI-compatible chat completions endpoint returned a 200 response whose `choices` array is empty, so the provider cannot extract any message. This usually indicates a server-side or gateway quirk (proxies, load balancers, content filters) rather than a malformed request. The provider defensively raises ValueError because downstream code assumes at least one choice exists.

Source

Thrown at deeptutor/services/llm/providers/open_ai.py:100

            or kwargs.pop("max_completion_tokens", None)
            or getattr(self.config, "max_tokens", 4096)
        )
        if isinstance(requested_max_tokens, (int, float, str)):
            max_tokens = int(requested_max_tokens)
        else:
            max_tokens = int(getattr(self.config, "max_tokens", 4096))
        kwargs.update(get_token_limit_kwargs(model, max_tokens))

        async def _call_api() -> TutorResponse:
            request_kwargs: dict[str, object] = dict(kwargs)
            response = await self.client.chat.completions.create(  # type: ignore[call-overload]
                model=model,
                messages=[{"role": "user", "content": prompt}],
                **request_kwargs,
            )

            if not response.choices:
                raise ValueError("API returned no choices in response")
            choice = response.choices[0]
            message = choice.message
            content = message.content or ""
            finish_reason = choice.finish_reason
            usage = response.usage.model_dump() if response.usage else {}
            raw_response = response.model_dump() if hasattr(response, "model_dump") else {}
            provider_label = (
                "azure" if isinstance(self.client, openai.AsyncAzureOpenAI) else "openai"
            )

            return TutorResponse(
                content=content,
                raw_response=raw_response,
                usage=usage,
                provider=provider_label,
                model=model,
                finish_reason=finish_reason,
                cost_estimate=self.calculate_cost(usage),

View on GitHub (pinned to 3e82f13042)

Solutions

1. Retry the request — transient empty-choices responses usually succeed on the next call (execute_with_retry may need a wrapper that also retries this ValueError).
2. Inspect raw_response / server logs to confirm whether a content filter or policy removed the choice.
3. Verify base_url and model name are correct for the endpoint you're hitting.
4. If using a proxy/gateway, upgrade or configure it to return proper error statuses instead of empty choices.

Example fix

# before
response = await self.client.chat.completions.create(...)
choice = response.choices[0]

# after
if not response.choices:
    raise ValueError("API returned no choices in response")
choice = response.choices[0]

# before calling:
if not provider.config.model:
    raise LLMConfigError("configure model first")  # unrelated, but cheap sanity check

try:
    resp = await provider.complete(prompt)
except ValueError as e:
    if "no choices" in str(e):
        resp = await provider.complete(prompt)  # or backoff-retry loop
    else:
        raise

When it happens

Trigger: Calling provider.complete() against an OpenAI-compatible endpoint (proxy, LiteLLM, vLLM, Azure gateway) that occasionally returns `{"choices": []}`; content-filtered or truncated responses; misconfigured base_url pointing at a non-chat-completions route.

Common situations: Self-hosted OpenAI-compatible servers with buggy streaming/non-streaming parity, API gateways that strip choices on policy rejection, rate-limiter middlewares that return empty bodies, or model names the backend doesn't recognize but fails softly on.

Related errors

• Cloud completion failed: no valid configuration
• No records provided
• Unknown agent kind: {kind!r}
• Both name and agent_kind are required.
• Unknown agent kind: {agent_kind!r}

AI-assisted analysis of HKUDS/DeepTutor@3e82f13042 (2026-08-27). Data as JSON: /api/errors/cc7cbefba81c0b0c. Report an issue: GitHub.