deepset-ai/haystack · error · ValueError

Cannot stream multiple responses, please set n=1.

Error message

Cannot stream multiple responses, please set n=1.

What it means

OpenAI's streaming API can only emit one completion per request. haystack enforces this by raising ValueError when a streaming_callback is supplied and generation_kwargs request n > 1 responses simultaneously.

Source

Thrown at haystack/components/generators/chat/openai.py:519

        return {"replies": completions}

    def _prepare_api_call(  # noqa: PLR0913
        self,
        *,
        messages: list[ChatMessage],
        streaming_callback: StreamingCallbackT | None = None,
        generation_kwargs: dict[str, Any] | None = None,
        tools: ToolsType | None = None,
        tools_strict: bool | None = None,
    ) -> dict[str, Any]:
        # update generation kwargs by merging with the generation kwargs passed to the run method
        generation_kwargs = {**self.generation_kwargs, **(generation_kwargs or {})}

        is_streaming = streaming_callback is not None
        num_responses = generation_kwargs.pop("n", 1)

        if is_streaming and num_responses > 1:
            raise ValueError("Cannot stream multiple responses, please set n=1.")
        response_format = generation_kwargs.pop("response_format", None)

        # adapt ChatMessage(s) to the format expected by the OpenAI API
        openai_formatted_messages = [message.to_openai_dict_format() for message in messages]

        flattened_tools = flatten_tools_or_toolsets(tools or self.tools)
        tools_strict = tools_strict if tools_strict is not None else self.tools_strict
        _check_duplicate_tool_names(flattened_tools)

        openai_tools = {}
        if flattened_tools:
            tool_definitions = []
            for t in flattened_tools:
                function_spec = {**t.tool_spec}
                if tools_strict:
                    function_spec["strict"] = True
                    function_spec["parameters"] = _make_schema_strict(function_spec["parameters"])
                tool_definitions.append({"type": "function", "function": function_spec})

View on GitHub (pinned to e318778c9b)

Solutions

1. Set n=1 (or remove n) when using streaming_callback
2. Run n parallel non-streaming calls if you need multiple candidates
3. Only pass streaming_callback for the streaming code path

Example fix

// before
gen.run(messages=msgs, generation_kwargs={"n": 3}, streaming_callback=cb)
// after
gen.run(messages=msgs, generation_kwargs={"n": 1}, streaming_callback=cb)

if streaming_callback is not None and kwargs.get("n", 1) > 1:
    kwargs["n"] = 1

try:
    gen.run(messages=msgs, generation_kwargs=gk, streaming_callback=cb)
except ValueError as e:
    if "Cannot stream multiple responses" in str(e):
        gk["n"] = 1
        gen.run(messages=msgs, generation_kwargs=gk, streaming_callback=cb)

When it happens

Trigger: Calling OpenAIChatGenerator.run(..., streaming_callback=cb) while generation_kwargs contains n=2 (or more), either set at init or merged via the per-call generation_kwargs.

Common situations: Upgrading code that previously requested multiple candidates non-interactively and adding streaming; a shared generation_kwargs dict carrying n>1 for both streaming and non-streaming paths.

Related errors

• A `ChatMessage` must contain at least one `TextContent`, `To
• 'response_fn' must return an assistant ChatMessage, got '{re
• Unsupported content type: {type(part)}
• For OpenAI compatibility, a `ChatMessage` with a `ToolCallRe
• Unsupported tool result: {result.result}

AI-assisted analysis of deepset-ai/haystack@e318778c9b (2026-08-30). Data as JSON: /api/errors/f56b9d8253cd2505. Report an issue: GitHub.