run-llama/llama_index · error · ValueError

Response must be a string or a generator. Found {type(respon

Error message

Response must be a string or a generator. Found {type(response_str)}

What it means

BaseSynthesizer._build_response dispatches on the runtime type of the LLM output: str becomes StreamingResponse, an async generator becomes AsyncStreamingResponse, and an instance of output_cls becomes PydanticResponse. Anything else (int, list, None, a custom object, or a structured output that does not match output_cls) hits this ValueError naming the offending type.

Source

Thrown at llama-index-core/llama_index/core/response_synthesizers/base.py:227

        if isinstance(response_str, Generator):
            return StreamingResponse(
                response_str,
                source_nodes=source_nodes,
                metadata=response_metadata,
            )
        if isinstance(response_str, AsyncGenerator):
            return AsyncStreamingResponse(
                response_str,
                source_nodes=source_nodes,
                metadata=response_metadata,
            )

        if self._output_cls is not None and isinstance(response_str, self._output_cls):
            return PydanticResponse(
                response_str, source_nodes=source_nodes, metadata=response_metadata
            )

        raise ValueError(
            f"Response must be a string or a generator. Found {type(response_str)}"
        )

    @dispatcher.span
    def synthesize(
        self,
        query: QueryTextType,
        nodes: List[NodeWithScore],
        additional_source_nodes: Optional[Sequence[NodeWithScore]] = None,
        **response_kwargs: Any,
    ) -> RESPONSE_TYPE:
        dispatcher.event(
            SynthesizeStartEvent(
                query=query,
            )
        )

        if len(nodes) == 0:

View on GitHub (pinned to afd0fef371)

Solutions

1. Make the custom LLM return str (e.g. response.text) for non-streaming calls and a generator/async generator for streaming calls.
2. With output_cls configured, ensure the LLM's structured-output adapter returns an instance of exactly that pydantic class (not a dict).
3. Log type(response_str) right before the failure to identify which LLM/adapter produced the bad type.
4. Wrap third-party model clients so they normalize output to CompletionResponse before it reaches the synthesizer.

Example fix

# before
class MyLLM(CustomLLM):
    def complete(self, prompt, **kwargs):
        return {"text": run_model(prompt)}  # dict -> ValueError

# after
from llama_index.core.llms import CompletionResponse
class MyLLM(CustomLLM):
    def complete(self, prompt, **kwargs):
        return CompletionResponse(text=run_model(prompt))

def normalize_response(resp, output_cls=None):
    if isinstance(resp, str) or hasattr(resp, "__anext__") or hasattr(resp, "__next__"):
        return resp
    if output_cls is not None and isinstance(resp, output_cls):
        return resp
    raise TypeError(f"LLM adapter returned unsupported type: {type(resp)}")

def is_valid_synthesis_output(resp, output_cls=None) -> bool:
    import types
    return (
        isinstance(resp, str)
        or isinstance(resp, (types.GeneratorType, types.AsyncGeneratorType))
        or (output_cls is not None and isinstance(resp, output_cls))
    )

When it happens

Trigger: Plugging in a custom LLM whose acomplete/astream_complete returns a non-string (e.g. raw dict or object) instead of str or a generator; using structured outputs (output_cls set) where the LLM adapter returns a dict rather than the pydantic model instance; a mocking layer in tests returning Mock objects.

Common situations: Custom LLM wrappers that forget to call .text on CompletionResponse; adapters for local models returning parsed JSON dicts; upgrading llama-index where response typing contracts tightened; test doubles leaking into synthesizer paths.

Related errors

• Got empty streaming response
• Expected ActionReasoningStep, got {reasoning_step}
• LLM only supports text inputs
• Invalid message content: {message.content!s}
• query and response must be provided

AI-assisted analysis of run-llama/llama_index@afd0fef371 (2026-08-15). Data as JSON: /api/errors/ad34e98063286846. Report an issue: GitHub.