microsoft/semantic-kernel · error · ServiceResponseException

Hugging Face completion failed

Error message

Hugging Face completion failed

What it means

Raised by HuggingFaceTextCompletion._inner_get_text_contents when the underlying Hugging Face pipeline call (self.generator(prompt, **settings.prepare_settings_dict())) throws any Exception. The connector wraps all failures into a ServiceResponseException, chaining the original via 'from e', so the real cause (OOM, model load error, bad input, device error) is in __cause__.

Source

Thrown at python/semantic_kernel/connectors/ai/hugging_face/services/hf_text_completion.py:109

    @override
    def get_prompt_execution_settings_class(self) -> type["PromptExecutionSettings"]:
        return HuggingFacePromptExecutionSettings

    @override
    @trace_text_completion(MODEL_PROVIDER_NAME)
    async def _inner_get_text_contents(
        self,
        prompt: str,
        settings: "PromptExecutionSettings",
    ) -> list[TextContent]:
        if not isinstance(settings, HuggingFacePromptExecutionSettings):
            settings = self.get_prompt_execution_settings_from_settings(settings)
        assert isinstance(settings, HuggingFacePromptExecutionSettings)  # nosec

        try:
            results = self.generator(prompt, **settings.prepare_settings_dict())
        except Exception as e:
            raise ServiceResponseException("Hugging Face completion failed") from e

        if isinstance(results, list):
            return [self._create_text_content(results, result) for result in results]
        return [self._create_text_content(results, results)]

    @override
    @trace_streaming_text_completion(MODEL_PROVIDER_NAME)
    async def _inner_get_streaming_text_contents(
        self,
        prompt: str,
        settings: "PromptExecutionSettings",
    ) -> AsyncGenerator[list[StreamingTextContent], Any]:
        if not isinstance(settings, HuggingFacePromptExecutionSettings):
            settings = self.get_prompt_execution_settings_from_settings(settings)
        assert isinstance(settings, HuggingFacePromptExecutionSettings)  # nosec

        if settings.num_return_sequences > 1:
            raise ServiceInvalidExecutionSettingsError(

View on GitHub (pinned to c028a0c7dc)

Solutions

1. Inspect the chained __cause__ (the original Exception) to find the true failure (CUDA OOM, model error, etc.).
2. For CUDA OOM, reduce max_new_tokens, use a smaller model, or move to CPU (device=-1).
3. Verify the task matches the model architecture and the model id is correct/downloaded.
4. Update/repair the model cache and ensure torch + transformers versions are compatible.

Example fix

# before
results = await svc.get_text_contents(prompt, settings)  # ServiceResponseException
# after: reduce memory / inspect cause
try:
    results = await svc.get_text_contents(prompt, settings)
except ServiceResponseException as e:
    logger.error('hf failed: %r', e.__cause__)
    settings.max_new_tokens = 64

assert isinstance(prompt, str) and prompt, 'prompt must be a non-empty string'
assert isinstance(settings, HuggingFacePromptExecutionSettings)

try:
    results = await svc.get_text_contents(prompt, settings)
except ServiceResponseException as e:
    logger.error('hf completion failed: %r', e.__cause__)
    raise

When it happens

Trigger: Running a local model that fails during generation: CUDA out-of-memory, an invalid generation config value, a model that failed to download/load, or an unsupported task/prompt. Any exception inside the pipeline call surfaces here.

Common situations: GPU OOM with large models. Mismatched task type and model (e.g. text-generation on an encoder model). Corrupt/incomplete model download. Invalid tokenizer/eos settings.

Related errors

• Hugging Face embeddings failed
• Agent Failure - Run terminated: {run.Status} [{run.Id}]: {ru
• Agent Failure - Run not created for thread: ${threadId}
• Invalid AgentId key: '{key}'. Must only contain ASCII charac
• Invalid AgentId type: '{type}'. Must be alphanumeric (a-z, 0

AI-assisted analysis of microsoft/semantic-kernel@c028a0c7dc (2026-08-13). Data as JSON: /api/errors/b19be7e2d30d95f0. Report an issue: GitHub.