run-llama/llama_index · error · ValueError

response_gen is only available for streaming responses. Set

Error message

response_gen is only available for streaming responses. Set is_dummy_stream=True if you still want a generator.

What it means

AgentChatResponse.response_gen is a fake-streaming property that yields whitespace-joined tokens of an already-complete response; it only exists so non-streaming responses (e.g. tool outputs) can mimic a token stream. Accessing it when is_dummy_stream is False raises ValueError, because a real streaming response carries its generator elsewhere (StreamingAgentChatResponse.response_gen) and this object has none.

Source

Thrown at llama-index-core/llama_index/core/chat_engine/types.py:84

    metadata: Optional[Dict[str, Any]] = None

    def set_source_nodes(self) -> None:
        if self.sources and not self.source_nodes:
            for tool_output in self.sources:
                if isinstance(tool_output.raw_output, (Response, StreamingResponse)):
                    self.source_nodes.extend(tool_output.raw_output.source_nodes)

    def __post_init__(self) -> None:
        self.set_source_nodes()

    def __str__(self) -> str:
        return self.response

    @property
    def response_gen(self) -> Generator[str, None, None]:
        """Used for fake streaming, i.e. with tool outputs."""
        if not self.is_dummy_stream:
            raise ValueError(
                "response_gen is only available for streaming responses. "
                "Set is_dummy_stream=True if you still want a generator."
            )

        for token in self.response.split(" "):
            yield token + " "
            time.sleep(0.1)

    async def async_response_gen(self) -> AsyncGenerator[str, None]:
        """Used for fake streaming, i.e. with tool outputs."""
        if not self.is_dummy_stream:
            raise ValueError(
                "response_gen is only available for streaming responses. "
                "Set is_dummy_stream=True if you still want a generator."
            )

        for token in self.response.split(" "):
            yield token + " "

View on GitHub (pinned to afd0fef371)

Solutions

Use stream_chat()/astream_chat() to get a StreamingAgentChatResponse whose response_gen is a true token generator
For non-streaming responses, print/consume resp.response directly (the full string)
If you deliberately want simulated token-by-token display of a finished response, construct AgentChatResponse(..., is_dummy_stream=True)

Example fix

# before
resp = engine.chat('hello')
for token in resp.response_gen:  # ValueError
    print(token, end='')

# after
resp = engine.stream_chat('hello')
for token in resp.response_gen:
    print(token, end='')
# or for non-streaming: print(resp.response)

Defensive patterns

Strategy: type-guard

Validate before calling

from llama_index.core.chat_engine import StreamingAgentChatResponse
if not isinstance(resp, StreamingAgentChatResponse):
    print(resp.response)  # full text; no generator available

Type guard

from llama_index.core.chat_engine import StreamingAgentChatResponse, AgentChatResponse

def has_real_stream(resp) -> bool:
    return isinstance(resp, StreamingAgentChatResponse)

# or for AgentChatResponse specifically:
def can_fake_stream(resp: AgentChatResponse) -> bool:
    return bool(resp.is_dummy_stream)

Try / catch

try:
    for token in resp.response_gen:
        print(token, end='')
except ValueError as e:
    if 'only available for streaming' in str(e):
        print(resp.response)
    else:
        raise

Prevention

Branch on the response type: iterate response_gen only for StreamingAgentChatResponse
Standardize endpoints on either chat() or stream_chat() instead of mixing both behind one consumer
If you need token-like output from a finished AgentChatResponse, create it with is_dummy_stream=True

When it happens

Trigger: resp = engine.chat('hi') (returns AgentChatResponse with is_dummy_stream=False) followed by resp.response_gen — the property's guard raises. Also hit in UI code that calls list(response.response_gen) on every response regardless of engine method.

Common situations: Unified render loops that iterate response_gen for both chat() and stream_chat() results; switching an endpoint from streaming to non-streaming while keeping the generator consumption; tool-calling engines where the final response is an AgentChatResponse.

Related errors

AI-assisted analysis of run-llama/llama_index@afd0fef371 (2026-08-15). Data as JSON: /api/errors/771d318447c0eba4. Report an issue: GitHub.