microsoft/semantic-kernel · error · ServiceResponseException

{type(self)} service failed to complete the chat

Error message

{type(self)} service failed to complete the chat

What it means

Raised as ServiceResponseException (chaining `ex`) when `self.client.chat.completions.create(**settings_dict)` throws any exception in `NvidiaHandler._send_chat_completion_request`. The block also merges settings.extra_body['nvext'] into the request for NVIDIA structured output, so a malformed nvext can contribute. The true upstream error is in `ex`/`__cause__`.

Source

Thrown at python/semantic_kernel/connectors/ai/nvidia/services/nvidia_handler.py:80

    async def _send_chat_completion_request(
        self, settings: NvidiaChatPromptExecutionSettings
    ) -> ChatCompletion | AsyncStream[Any]:
        """Send a request to the NVIDIA chat completion endpoint."""
        try:
            settings_dict = settings.prepare_settings_dict()

            # Handle structured output if nvext is present in extra_body
            if settings.extra_body and "nvext" in settings.extra_body:
                if "extra_body" not in settings_dict:
                    settings_dict["extra_body"] = {}
                settings_dict["extra_body"]["nvext"] = settings.extra_body["nvext"]

            response = await self.client.chat.completions.create(**settings_dict)
            self.store_usage(response)
            return response
        except Exception as ex:
            raise ServiceResponseException(
                f"{type(self)} service failed to complete the chat",
                ex,
            ) from ex

    def store_usage(
        self,
        response: ChatCompletion
        | Completion
        | AsyncStream[ChatCompletionChunk]
        | AsyncStream[Completion]
        | CreateEmbeddingResponse,
    ):
        """Store the usage information from the response."""
        if not isinstance(response, AsyncStream) and response.usage:
            logger.info(f"OpenAI usage: {response.usage}")
            self.prompt_tokens += response.usage.prompt_tokens
            self.total_tokens += response.usage.total_tokens
            if hasattr(response.usage, "completion_tokens"):

View on GitHub (pinned to c028a0c7dc)

Solutions

1. Inspect `e.__cause__` for the exact upstream status/message.
2. Verify ai_model_id is a valid model id for the NVIDIA base_url endpoint.
3. If using extra_body/nvext structured output, validate its shape matches NVIDIA's nvext schema; remove it to isolate the cause.
4. Strip unsupported params from the request or move them into extra_body.
5. Retry on 429/timeout; confirm NVIDIA_API_KEY validity.

Example fix

# before
resp = await svc.get_chat_message_contents(chat_history, settings)

# after
try:
    resp = await svc.get_chat_message_contents(chat_history, settings)
except ServiceResponseException as e:
    raise RuntimeError(f"NVIDIA chat upstream error: {e.__cause__!r}") from e

assert svc.ai_model_id, 'ai_model_id must be set for NVIDIA chat'
if getattr(settings, 'extra_body', None) and 'nvext' in settings.extra_body:
    assert isinstance(settings.extra_body['nvext'], dict), 'nvext must be a dict'

from semantic_kernel.exceptions import ServiceResponseException

def is_nvidia_chat_error(e: BaseException) -> bool:
    return isinstance(e, ServiceResponseException) and 'failed to complete the chat' in str(e)

from semantic_kernel.exceptions import ServiceResponseException
try:
    resp = await svc.get_chat_message_contents(chat_history, settings)
except ServiceResponseException as e:
    cause = e.__cause__
    raise RuntimeError(f'NVIDIA chat upstream error: {cause!r}') from cause

When it happens

Trigger: NVIDIA chat completion where the OpenAI-style client raises: 401 auth, 404 unknown model, 400/422 from unsupported params or a malformed nvext extra_body, 429 rate limit, or network/timeout. Also if settings_dict contains keys the NVIDIA endpoint rejects.

Common situations: ai_model_id not deployed on the NVIDIA endpoint, passing OpenAI-specific params NVIDIA rejects, malformed structured-output nvext payload, expired NVIDIA_API_KEY, transient rate limit.

Related errors

• {type(self)} service failed to generate embeddings
• {type(self)} service failed to complete the embedding reques
• The NVIDIA API key is required.
• Unable to transform result into {typeof(TOutput).Name}
• {type(self)} service failed to complete the prompt

AI-assisted analysis of microsoft/semantic-kernel@c028a0c7dc (2026-08-13). Data as JSON: /api/errors/e77a6286ab63a0bd. Report an issue: GitHub.