{"record":{"id":"ad34e98063286846","repo":"run-llama/llama_index","slug":"response-must-be-a-string-or-a-generator-found-t","errorCode":null,"errorMessage":"Response must be a string or a generator. Found {type(response_str)}","messagePattern":"Response must be a string or a generator\\. Found (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"llama-index-core/llama_index/core/response_synthesizers/base.py","lineNumber":227,"sourceCode":"        if isinstance(response_str, Generator):\n            return StreamingResponse(\n                response_str,\n                source_nodes=source_nodes,\n                metadata=response_metadata,\n            )\n        if isinstance(response_str, AsyncGenerator):\n            return AsyncStreamingResponse(\n                response_str,\n                source_nodes=source_nodes,\n                metadata=response_metadata,\n            )\n\n        if self._output_cls is not None and isinstance(response_str, self._output_cls):\n            return PydanticResponse(\n                response_str, source_nodes=source_nodes, metadata=response_metadata\n            )\n\n        raise ValueError(\n            f\"Response must be a string or a generator. Found {type(response_str)}\"\n        )\n\n    @dispatcher.span\n    def synthesize(\n        self,\n        query: QueryTextType,\n        nodes: List[NodeWithScore],\n        additional_source_nodes: Optional[Sequence[NodeWithScore]] = None,\n        **response_kwargs: Any,\n    ) -> RESPONSE_TYPE:\n        dispatcher.event(\n            SynthesizeStartEvent(\n                query=query,\n            )\n        )\n\n        if len(nodes) == 0:","sourceCodeStart":209,"sourceCodeEnd":245,"githubUrl":"https://github.com/run-llama/llama_index/blob/afd0fef371831f9bda13e5af7167cf4e981278ab/llama-index-core/llama_index/core/response_synthesizers/base.py#L209-L245","documentation":"BaseSynthesizer._build_response dispatches on the runtime type of the LLM output: str becomes StreamingResponse, an async generator becomes AsyncStreamingResponse, and an instance of output_cls becomes PydanticResponse. Anything else (int, list, None, a custom object, or a structured output that does not match output_cls) hits this ValueError naming the offending type.","triggerScenarios":"Plugging in a custom LLM whose acomplete/astream_complete returns a non-string (e.g. raw dict or object) instead of str or a generator; using structured outputs (output_cls set) where the LLM adapter returns a dict rather than the pydantic model instance; a mocking layer in tests returning Mock objects.","commonSituations":"Custom LLM wrappers that forget to call .text on CompletionResponse; adapters for local models returning parsed JSON dicts; upgrading llama-index where response typing contracts tightened; test doubles leaking into synthesizer paths.","solutions":["Make the custom LLM return str (e.g. response.text) for non-streaming calls and a generator/async generator for streaming calls.","With output_cls configured, ensure the LLM's structured-output adapter returns an instance of exactly that pydantic class (not a dict).","Log type(response_str) right before the failure to identify which LLM/adapter produced the bad type.","Wrap third-party model clients so they normalize output to CompletionResponse before it reaches the synthesizer."],"exampleFix":"# before\nclass MyLLM(CustomLLM):\n    def complete(self, prompt, **kwargs):\n        return {\"text\": run_model(prompt)}  # dict -> ValueError\n\n# after\nfrom llama_index.core.llms import CompletionResponse\nclass MyLLM(CustomLLM):\n    def complete(self, prompt, **kwargs):\n        return CompletionResponse(text=run_model(prompt))","handlingStrategy":"type-guard","validationCode":"def normalize_response(resp, output_cls=None):\n    if isinstance(resp, str) or hasattr(resp, \"__anext__\") or hasattr(resp, \"__next__\"):\n        return resp\n    if output_cls is not None and isinstance(resp, output_cls):\n        return resp\n    raise TypeError(f\"LLM adapter returned unsupported type: {type(resp)}\")","typeGuard":"def is_valid_synthesis_output(resp, output_cls=None) -> bool:\n    import types\n    return (\n        isinstance(resp, str)\n        or isinstance(resp, (types.GeneratorType, types.AsyncGeneratorType))\n        or (output_cls is not None and isinstance(resp, output_cls))\n    )","tryCatchPattern":null,"preventionTips":["Custom LLMs must return CompletionResponse/ChatResponse (or .text), never raw dicts.","For structured output, ensure adapters instantiate the exact output_cls pydantic model.","Wrap external model clients once, at the boundary, with output normalization."],"tags":["llm","custom-llm","type-error","response-synthesizer"],"backgroundTag":null,"analyzedSha":"afd0fef371831f9bda13e5af7167cf4e981278ab","analyzedAt":"2026-08-15T05:42:58.429Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}