microsoft/semantic-kernel · error · ServiceInvalidExecutionSettingsError

HuggingFace TextIteratorStreamer does not stream multiple re

Error message

HuggingFace TextIteratorStreamer does not stream multiple responses in a parsable format. If you need multiple responses, please use the complete method.

What it means

Raised by HuggingFaceTextCompletion._inner_get_streaming_text_contents when HuggingFacePromptExecutionSettings.num_return_sequences is greater than 1. The TextIteratorStreamer cannot emit multiple sequences in a parseable way, so streaming with multiple return sequences is rejected up front before any thread is started.

Source

Thrown at python/semantic_kernel/connectors/ai/hugging_face/services/hf_text_completion.py:127

            raise ServiceResponseException("Hugging Face completion failed") from e

        if isinstance(results, list):
            return [self._create_text_content(results, result) for result in results]
        return [self._create_text_content(results, results)]

    @override
    @trace_streaming_text_completion(MODEL_PROVIDER_NAME)
    async def _inner_get_streaming_text_contents(
        self,
        prompt: str,
        settings: "PromptExecutionSettings",
    ) -> AsyncGenerator[list[StreamingTextContent], Any]:
        if not isinstance(settings, HuggingFacePromptExecutionSettings):
            settings = self.get_prompt_execution_settings_from_settings(settings)
        assert isinstance(settings, HuggingFacePromptExecutionSettings)  # nosec

        if settings.num_return_sequences > 1:
            raise ServiceInvalidExecutionSettingsError(
                "HuggingFace TextIteratorStreamer does not stream multiple responses in a parsable format."
                " If you need multiple responses, please use the complete method.",
            )
        try:
            streamer = TextIteratorStreamer(AutoTokenizer.from_pretrained(self.ai_model_id))
            # See https://github.com/huggingface/transformers/blob/main/src/transformers/generation/streamers.py#L159
            thread = Thread(
                target=self.generator, args={prompt}, kwargs=settings.prepare_settings_dict(streamer=streamer)
            )
            thread.start()

            for new_text in streamer:
                yield [
                    StreamingTextContent(
                        choice_index=0, inner_content=new_text, text=new_text, ai_model_id=self.ai_model_id
                    )
                ]

View on GitHub (pinned to c028a0c7dc)

Solutions

1. Set num_return_sequences=1 when using the streaming completion method.
2. If you need multiple responses, call the non-streaming complete method (_inner_get_text_contents) instead.
3. Branch your code: streaming path forces num_return_sequences=1, batch path allows >1.

Example fix

# before
settings = HuggingFacePromptExecutionSettings(num_return_sequences=3)
async for chunk in svc.get_streaming_text_contents(prompt, settings): ...
# after
settings = HuggingFacePromptExecutionSettings(num_return_sequences=1)
async for chunk in svc.get_streaming_text_contents(prompt, settings): ...

if streaming:
    assert settings.num_return_sequences <= 1, 'Streaming requires num_return_sequences=1'

When it happens

Trigger: Requesting streaming text completion while setting num_return_sequences > 1 on HuggingFacePromptExecutionSettings.

Common situations: Wanting several completion candidates (num_return_sequences=3) and calling the streaming API instead of the complete method. Carrying over multi-candidate settings into a streaming call.

Related errors

• The streaming configuration must be null for non-streaming r
• The streaming configuration must have StreamFinalResponse se
• Auto invocation of tool calls may only be used with a single
• Failed to validate Google AI settings: {e}
• Failed to validate Google AI settings: {e}

AI-assisted analysis of microsoft/semantic-kernel@c028a0c7dc (2026-08-13). Data as JSON: /api/errors/4f9a25e582169879. Report an issue: GitHub.