zylon-ai/private-gpt · error · NotImplementedError

Streaming responses are not yet implemented for summarizatio

Error message

Streaming responses are not yet implemented for summarization

What it means

execute_summarize explicitly rejects StreamingResponse: the summarization workflow requires a complete response to sanitize and return as one SummarizeResultEvent, so a query engine configured for streaming (response_mode='compact'/'streaming', streaming=True, or a streaming synthesizer) raises NotImplementedError by design.

Source

Thrown at private_gpt/components/workflows/others/summary.py:241

        if ev.output_cls and isinstance(response, PydanticResponse):
            if not response.response:
                raise ValueError("No response was generated")

            return SummarizeResultEvent(
                output_obj=response.response,
            )

        if isinstance(response, Response):
            summary = response.response or ev.empty_response_fallback or ""
            if not summary:
                raise ValueError("No summary was generated")

            sanitized = MarkdownHelper.sanitize_markdown(summary)
            return SummarizeResultEvent(summary=sanitized or summary)

        elif isinstance(response, StreamingResponse):
            raise NotImplementedError(
                "Streaming responses are not yet implemented for summarization"
            )

        raise TypeError(f"Unsupported response type: {type(response)}")

View on GitHub (pinned to 4a030776a3)

Solutions

1. Configure a separate non-streaming query engine for the SummarizeWorkflow (streaming=False, complete responses).
2. If streaming summaries are required, collect the stream to completion (response.response_gen fully consumed -> Response) before feeding the workflow, or implement a streaming branch in execute_summarize.
3. Check CustomSynthesisBuilder / synthesize() settings used by summary_query_engine to ensure they produce Response, not StreamingResponse.

Example fix

# before
engine = summary_query_engine(..., streaming=True)

# after
engine = summary_query_engine(..., streaming=False)

engine = summary_query_engine(..., streaming=False)
assert not getattr(engine, 'streaming', False), 'summarization engine must not stream'

def is_complete_response(r: object) -> bool:
    return isinstance(r, Response) or isinstance(r, PydanticResponse)

except NotImplementedError as e:
    if 'Streaming' in str(e):
        response = await collect_stream_to_response(response_gen)
    else:
        raise

When it happens

Trigger: Constructing the summary query engine with streaming=True or a CustomSynthesisBuilder that returns StreamingResponse, then running the SummarizeWorkflow; using a shared engine instance configured for chat streaming inside the summarize path.

Common situations: Reusing the chat-ingestion engine (streaming) for summarization; upgrading pipelines where the default engine became streaming; wiring a StreamingResponse-capable retriever/synthesizer into the summary workflow.

Related errors

• Unsupported streaming provider: {settings.stream.broker}
• No summary was generated
• scheduler.chat.mode={self.scheduler.chat.mode!r} requires st
• Invalid system item in list: {item}
• Unknown condense strategy: {strategy}

AI-assisted analysis of zylon-ai/private-gpt@4a030776a3 (2026-08-15). Data as JSON: /api/errors/89be642491da99f9. Report an issue: GitHub.