{"record":{"id":"b19be7e2d30d95f0","repo":"microsoft/semantic-kernel","slug":"hugging-face-completion-failed","errorCode":null,"errorMessage":"Hugging Face completion failed","messagePattern":"Hugging Face completion failed","errorType":"exception","errorClass":"ServiceResponseException","httpStatus":null,"severity":"error","filePath":"python/semantic_kernel/connectors/ai/hugging_face/services/hf_text_completion.py","lineNumber":109,"sourceCode":"    @override\n    def get_prompt_execution_settings_class(self) -> type[\"PromptExecutionSettings\"]:\n        return HuggingFacePromptExecutionSettings\n\n    @override\n    @trace_text_completion(MODEL_PROVIDER_NAME)\n    async def _inner_get_text_contents(\n        self,\n        prompt: str,\n        settings: \"PromptExecutionSettings\",\n    ) -> list[TextContent]:\n        if not isinstance(settings, HuggingFacePromptExecutionSettings):\n            settings = self.get_prompt_execution_settings_from_settings(settings)\n        assert isinstance(settings, HuggingFacePromptExecutionSettings)  # nosec\n\n        try:\n            results = self.generator(prompt, **settings.prepare_settings_dict())\n        except Exception as e:\n            raise ServiceResponseException(\"Hugging Face completion failed\") from e\n\n        if isinstance(results, list):\n            return [self._create_text_content(results, result) for result in results]\n        return [self._create_text_content(results, results)]\n\n    @override\n    @trace_streaming_text_completion(MODEL_PROVIDER_NAME)\n    async def _inner_get_streaming_text_contents(\n        self,\n        prompt: str,\n        settings: \"PromptExecutionSettings\",\n    ) -> AsyncGenerator[list[StreamingTextContent], Any]:\n        if not isinstance(settings, HuggingFacePromptExecutionSettings):\n            settings = self.get_prompt_execution_settings_from_settings(settings)\n        assert isinstance(settings, HuggingFacePromptExecutionSettings)  # nosec\n\n        if settings.num_return_sequences > 1:\n            raise ServiceInvalidExecutionSettingsError(","sourceCodeStart":91,"sourceCodeEnd":127,"githubUrl":"https://github.com/microsoft/semantic-kernel/blob/c028a0c7dc4f0814cdcbaba9d998f187a41197bf/python/semantic_kernel/connectors/ai/hugging_face/services/hf_text_completion.py#L91-L127","documentation":"Raised by HuggingFaceTextCompletion._inner_get_text_contents when the underlying Hugging Face pipeline call (self.generator(prompt, **settings.prepare_settings_dict())) throws any Exception. The connector wraps all failures into a ServiceResponseException, chaining the original via 'from e', so the real cause (OOM, model load error, bad input, device error) is in __cause__.","triggerScenarios":"Running a local model that fails during generation: CUDA out-of-memory, an invalid generation config value, a model that failed to download/load, or an unsupported task/prompt. Any exception inside the pipeline call surfaces here.","commonSituations":"GPU OOM with large models. Mismatched task type and model (e.g. text-generation on an encoder model). Corrupt/incomplete model download. Invalid tokenizer/eos settings.","solutions":["Inspect the chained __cause__ (the original Exception) to find the true failure (CUDA OOM, model error, etc.).","For CUDA OOM, reduce max_new_tokens, use a smaller model, or move to CPU (device=-1).","Verify the task matches the model architecture and the model id is correct/downloaded.","Update/repair the model cache and ensure torch + transformers versions are compatible."],"exampleFix":"# before\nresults = await svc.get_text_contents(prompt, settings)  # ServiceResponseException\n# after: reduce memory / inspect cause\ntry:\n    results = await svc.get_text_contents(prompt, settings)\nexcept ServiceResponseException as e:\n    logger.error('hf failed: %r', e.__cause__)\n    settings.max_new_tokens = 64","handlingStrategy":"try-catch","validationCode":"assert isinstance(prompt, str) and prompt, 'prompt must be a non-empty string'\nassert isinstance(settings, HuggingFacePromptExecutionSettings)","typeGuard":null,"tryCatchPattern":"try:\n    results = await svc.get_text_contents(prompt, settings)\nexcept ServiceResponseException as e:\n    logger.error('hf completion failed: %r', e.__cause__)\n    raise","preventionTips":["Always log e.__cause__ to find the true failure (OOM, model error).","Validate prompt and settings before calling.","Monitor GPU memory and reduce max_new_tokens under pressure."],"tags":["hugging-face","text-completion","runtime","gpu"],"backgroundTag":null,"analyzedSha":"c028a0c7dc4f0814cdcbaba9d998f187a41197bf","analyzedAt":"2026-08-13T13:48:05.040Z","schemaVersion":2},"datasetVersion":"2026-08-13T14:17:21.547Z"}