huggingface/smolagents · error · RuntimeError

Unexpected API response: model '{self.model_id}' returned no

Error message

Unexpected API response: model '{self.model_id}' returned no choices.  This may indicate a possible API or upstream issue. Response details: {response.model_dump()}

What it means

LiteLLMModel.generate raises this RuntimeError when the LiteLLM completion call succeeds at the transport level but the returned response object contains an empty `choices` list. smolagents expects at least one choice to extract `response.choices[0].message.content`, so an empty list means it cannot proceed. It dumps the full response via `model_dump()` to help diagnose whether the upstream provider returned an error payload or a malformed body.

Source

Thrown at src/smolagents/models.py:1290

        **kwargs,
    ) -> ChatMessage:
        completion_kwargs = self._prepare_completion_kwargs(
            messages=messages,
            stop_sequences=stop_sequences,
            response_format=response_format,
            tools_to_call_from=tools_to_call_from,
            model=self.model_id,
            api_base=self.api_base,
            api_key=self.api_key,
            convert_images_to_image_urls=True,
            custom_role_conversions=self.custom_role_conversions,
            **kwargs,
        )
        self._apply_rate_limit()
        response = self.retryer(self.client.completion, **completion_kwargs)

        if not response.choices:
            raise RuntimeError(
                f"Unexpected API response: model '{self.model_id}' returned no choices. "
                " This may indicate a possible API or upstream issue. "
                f"Response details: {response.model_dump()}"
            )
        content = response.choices[0].message.content
        if stop_sequences is not None and not self.supports_stop_parameter:
            content = remove_content_after_stop_sequences(content, stop_sequences)
        return ChatMessage(
            role=response.choices[0].message.role,
            content=content,
            tool_calls=response.choices[0].message.tool_calls,
            raw=response,
            token_usage=TokenUsage(
                input_tokens=response.usage.prompt_tokens,
                output_tokens=response.usage.completion_tokens,
            ),
        )

View on GitHub (pinned to 30bb116109)

Solutions

1. Inspect the Response details in the message: if it contains an error field, address that upstream error (auth, rate limit, content filter).
2. Verify model_id and api_base are correct and that the model is actually deployed/served.
3. Reproduce with `litellm.completion(...)` directly to see whether LiteLLM or the provider drops the choices.
4. Upgrade/downgrade litellm to a known-good version; empty-choices behavior varies across releases.
5. Wrap agent/model calls in retry logic (smolagents models support a retryer) to ride out transient upstream failures.

Example fix

# before
model = LiteLLMModel(model_id="my-model", api_base="http://localhost:8000")
out = model([{"role": "user", "content": "hi"}])  # RuntimeError: returned no choices

# after
import litellm
resp = litellm.completion(model="my-model", api_base="http://localhost:8000", messages=[{"role":"user","content":"hi"}])
print(resp)  # inspect what the endpoint actually returns; fix deployment/auth accordingly

import litellm
resp = litellm.completion(model=model_id, api_base=base_url, messages=[{"role":"user","content":"ping"}])
assert resp.choices, f"endpoint returned no choices: {resp}"

def has_choices(resp) -> bool:
    return bool(getattr(resp, "choices", None))

try:
    out = model(messages)
except RuntimeError as e:
    if "returned no choices" in str(e):
        logger.error("Upstream issue, inspect response details: %s", e)
        out = model(messages)  # or back off / switch model
    else:
        raise

When it happens

Trigger: Calling model(...) or generate(...) on LiteLLMModel where the provider (e.g. an OpenAI-compatible endpoint, Azure, or a proxied model) returns 200 with no choices; LiteLLM sometimes swallows upstream errors (rate limits, content filter blocks, provider outages) into an empty-choices response instead of raising.

Common situations: Misconfigured base_url/model_id pointing at an endpoint that returns an empty body; upstream provider outage or content moderation blocking; LiteLLM version changes altering error propagation; using a model name the router/deployment does not actually serve.

Related errors

• Error while generating output: {e}
• Error in generating model output: {e}
• No content or tool calls in event: {event}
• Error during jinja template rendering: {type(e).__name__}: {
• Cannot specify both 'messages' and 'steps' parameters. Use '

AI-assisted analysis of huggingface/smolagents@30bb116109 (2026-08-28). Data as JSON: /api/errors/2506c00330399727. Report an issue: GitHub.