run-llama/llama_index · error · ValueError

Unable to stream in Accumulate response mode

Error message

Unable to stream in Accumulate response mode

What it means

Accumulate response mode runs a separate LLM call per text chunk and joins results, which has no meaningful single streaming output. aget_response therefore raises ValueError if the synthesizer was constructed with streaming=True. The same guard exists in the sync get_response and both *_from_messages variants (lines 158, 182, 202).

Source

Thrown at llama-index-core/llama_index/core/response_synthesizers/accumulate.py:136

            return [
                predictor(
                    self._output_cls,
                    template,
                    **self._make_prompt_kwargs(c),
                    **response_kwargs,
                )
                for c in repacked
            ]

    async def aget_response(
        self,
        query_str: str,
        text_chunks: Sequence[str],
        separator: str = "\n---------------------\n",
        **response_kwargs: Any,
    ) -> RESPONSE_TEXT_TYPE:
        if self._streaming:
            raise ValueError("Unable to stream in Accumulate response mode")

        tasks = [
            self._give_responses(
                query_str, text_chunk, use_async=True, **response_kwargs
            )
            for text_chunk in text_chunks
        ]

        flattened_tasks = self.flatten_list(tasks)
        outputs = await asyncio.gather(*flattened_tasks)

        return self._format_response(outputs, separator)

    def get_response(
        self,
        query_str: str,
        text_chunks: Sequence[str],
        separator: str = "\n---------------------\n",

View on GitHub (pinned to afd0fef371)

Solutions

1. Set streaming=False (or omit it) for the Accumulate synthesizer: get_response_def(..., response_mode='accumulate', streaming=False).
2. If you need streaming, use a mode that supports it, such as 'compact' or 'refine', or stream chunk results yourself by processing each chunk's response as it completes.
3. Use AccumulateResponseSynthesizer(streaming=False) explicitly when constructing the synthesizer by hand.

Example fix

# before
query_engine = index.as_query_engine(
    response_mode="accumulate", streaming=True
)

# after
query_engine = index.as_query_engine(
    response_mode="accumulate", streaming=False
)

STREAMABLE_MODES = {"compact", "refine", "tree_summarize", "simple_summarize"}
if response_mode == "accumulate":
    assert not streaming, "accumulate mode cannot stream"

When it happens

Trigger: Building a query engine with response_mode='accumulate' and streaming=True (via ResponseSynthesizer or QueryEngineArgs), then awaiting aget_response / astream_query; copying streaming chat-engine configuration onto an accumulate-mode engine; passing streaming=True through a RetrieverQueryEngine.with_response_mode helper.

Common situations: Reusing a streaming template config across different response modes; LLM+RAG demos combining accumulate mode with chat-style streaming UX; setting streaming globally on Settings and then switching an engine to accumulate.

Related errors

• Streaming is not enabled. Please use achat() instead.
• achat_stream is None. Cannot asynchronously write to history
• achat_stream is None!
• astream_complete is not supported by default.
• AsyncStreamingResponse not supported in sync code.

AI-assisted analysis of run-llama/llama_index@afd0fef371 (2026-08-15). Data as JSON: /api/errors/cbf06d02373c8c66. Report an issue: GitHub.