{"record":{"id":"4f9a25e582169879","repo":"microsoft/semantic-kernel","slug":"huggingface-textiteratorstreamer-does-not-stream-m","errorCode":null,"errorMessage":"HuggingFace TextIteratorStreamer does not stream multiple responses in a parsable format. If you need multiple responses, please use the complete method.","messagePattern":"HuggingFace TextIteratorStreamer does not stream multiple responses in a parsable format\\. If you need multiple responses, please use the complete method\\.","errorType":"exception","errorClass":"ServiceInvalidExecutionSettingsError","httpStatus":null,"severity":"error","filePath":"python/semantic_kernel/connectors/ai/hugging_face/services/hf_text_completion.py","lineNumber":127,"sourceCode":"            raise ServiceResponseException(\"Hugging Face completion failed\") from e\n\n        if isinstance(results, list):\n            return [self._create_text_content(results, result) for result in results]\n        return [self._create_text_content(results, results)]\n\n    @override\n    @trace_streaming_text_completion(MODEL_PROVIDER_NAME)\n    async def _inner_get_streaming_text_contents(\n        self,\n        prompt: str,\n        settings: \"PromptExecutionSettings\",\n    ) -> AsyncGenerator[list[StreamingTextContent], Any]:\n        if not isinstance(settings, HuggingFacePromptExecutionSettings):\n            settings = self.get_prompt_execution_settings_from_settings(settings)\n        assert isinstance(settings, HuggingFacePromptExecutionSettings)  # nosec\n\n        if settings.num_return_sequences > 1:\n            raise ServiceInvalidExecutionSettingsError(\n                \"HuggingFace TextIteratorStreamer does not stream multiple responses in a parsable format.\"\n                \" If you need multiple responses, please use the complete method.\",\n            )\n        try:\n            streamer = TextIteratorStreamer(AutoTokenizer.from_pretrained(self.ai_model_id))\n            # See https://github.com/huggingface/transformers/blob/main/src/transformers/generation/streamers.py#L159\n            thread = Thread(\n                target=self.generator, args={prompt}, kwargs=settings.prepare_settings_dict(streamer=streamer)\n            )\n            thread.start()\n\n            for new_text in streamer:\n                yield [\n                    StreamingTextContent(\n                        choice_index=0, inner_content=new_text, text=new_text, ai_model_id=self.ai_model_id\n                    )\n                ]\n","sourceCodeStart":109,"sourceCodeEnd":145,"githubUrl":"https://github.com/microsoft/semantic-kernel/blob/c028a0c7dc4f0814cdcbaba9d998f187a41197bf/python/semantic_kernel/connectors/ai/hugging_face/services/hf_text_completion.py#L109-L145","documentation":"Raised by HuggingFaceTextCompletion._inner_get_streaming_text_contents when HuggingFacePromptExecutionSettings.num_return_sequences is greater than 1. The TextIteratorStreamer cannot emit multiple sequences in a parseable way, so streaming with multiple return sequences is rejected up front before any thread is started.","triggerScenarios":"Requesting streaming text completion while setting num_return_sequences > 1 on HuggingFacePromptExecutionSettings.","commonSituations":"Wanting several completion candidates (num_return_sequences=3) and calling the streaming API instead of the complete method. Carrying over multi-candidate settings into a streaming call.","solutions":["Set num_return_sequences=1 when using the streaming completion method.","If you need multiple responses, call the non-streaming complete method (_inner_get_text_contents) instead.","Branch your code: streaming path forces num_return_sequences=1, batch path allows >1."],"exampleFix":"# before\nsettings = HuggingFacePromptExecutionSettings(num_return_sequences=3)\nasync for chunk in svc.get_streaming_text_contents(prompt, settings): ...\n# after\nsettings = HuggingFacePromptExecutionSettings(num_return_sequences=1)\nasync for chunk in svc.get_streaming_text_contents(prompt, settings): ...","handlingStrategy":"validation","validationCode":"if streaming:\n    assert settings.num_return_sequences <= 1, 'Streaming requires num_return_sequences=1'","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Force num_return_sequences=1 on the streaming path.","Use the complete (non-streaming) method when you need multiple candidates.","Branch settings by call mode."],"tags":["hugging-face","streaming","settings","validation"],"backgroundTag":null,"analyzedSha":"c028a0c7dc4f0814cdcbaba9d998f187a41197bf","analyzedAt":"2026-08-13T13:48:05.040Z","schemaVersion":2},"datasetVersion":"2026-08-13T14:17:21.547Z"}