microsoft/semantic-kernel · error · ServiceResponseException

{type(self)} service failed to generate embeddings

Error message

{type(self)} service failed to generate embeddings

What it means

Raised as ServiceResponseException (chaining `ex`) when `self.client.embeddings.create(**settings.prepare_settings_dict())` throws any exception in `NvidiaHandler._send_embedding_request`. It is the NVIDIA embeddings catch-all around the OpenAI-compatible client; the real cause is in `ex`/`__cause__`. store_usage and result parsing only run if the call succeeds.

Source

Thrown at python/semantic_kernel/connectors/ai/nvidia/services/nvidia_handler.py:58

        if self.ai_model_type == NvidiaModelTypes.EMBEDDING:
            assert isinstance(settings, NvidiaEmbeddingPromptExecutionSettings)  # nosec
            return await self._send_embedding_request(settings)
        if self.ai_model_type == NvidiaModelTypes.CHAT:
            assert isinstance(settings, NvidiaChatPromptExecutionSettings)  # nosec
            return await self._send_chat_completion_request(settings)

        raise NotImplementedError(f"Model type {self.ai_model_type} is not supported")

    async def _send_embedding_request(self, settings: NvidiaEmbeddingPromptExecutionSettings) -> list[Any]:
        """Send a request to the OpenAI embeddings endpoint."""
        try:
            # unsupported parameters are internally excluded from main dict and added to extra_body
            response = await self.client.embeddings.create(**settings.prepare_settings_dict())

            self.store_usage(response)
            return [x.embedding for x in response.data]
        except Exception as ex:
            raise ServiceResponseException(
                f"{type(self)} service failed to generate embeddings",
                ex,
            ) from ex

    async def _send_chat_completion_request(
        self, settings: NvidiaChatPromptExecutionSettings
    ) -> ChatCompletion | AsyncStream[Any]:
        """Send a request to the NVIDIA chat completion endpoint."""
        try:
            settings_dict = settings.prepare_settings_dict()

            # Handle structured output if nvext is present in extra_body
            if settings.extra_body and "nvext" in settings.extra_body:
                if "extra_body" not in settings_dict:
                    settings_dict["extra_body"] = {}
                settings_dict["extra_body"]["nvext"] = settings.extra_body["nvext"]

            response = await self.client.chat.completions.create(**settings_dict)

View on GitHub (pinned to c028a0c7dc)

Solutions

  1. Inspect `e.__cause__` for the upstream HTTP status/message.
  2. Ensure ai_model_id is an NVIDIA embedding model (e.g. 'nvidia/nv-embedqa-e5-v5', 'NV-Embed-QA').
  3. Filter empty/oversized inputs from the texts list before calling.
  4. Retry on 429/timeout with backoff; verify NVIDIA_API_KEY is valid.

Example fix

# before
vecs = await svc.generate_embeddings(texts)

# after
try:
    vecs = await svc.generate_embeddings([t for t in texts if t.strip()])
except ServiceResponseException as e:
    raise RuntimeError(f"NVIDIA embedding upstream error: {e.__cause__!r}") from e
Defensive patterns

Strategy: retry

Validate before calling

assert texts and all(t.strip() for t in texts), 'embed non-empty texts only'
assert svc.ai_model_id and 'embed' in svc.ai_model_id.lower(), 'use an NVIDIA embedding model id'

Type guard

from semantic_kernel.exceptions import ServiceResponseException

def is_nvidia_embed_error(e: BaseException) -> bool:
    return isinstance(e, ServiceResponseException) and 'failed to generate embeddings' in str(e)

Try / catch

import asyncio
from semantic_kernel.exceptions import ServiceResponseException
async def embed_retry(svc, texts, attempts=3):
    for i in range(attempts):
        try:
            return await svc.generate_embeddings(texts)
        except ServiceResponseException as e:
            if getattr(e.__cause__, 'status_code', None) in (429, 503):
                await asyncio.sleep(2 ** i); continue
            raise
    raise RuntimeError('embeddings failed after retries')

Prevention

When it happens

Trigger: Generating NVIDIA embeddings where the OpenAI-style client raises: 401 auth, 404 unknown embedding model, 400/422 from unsupported parameters that were not excluded into extra_body, 429 rate limit, or network/timeout.

Common situations: ai_model_id is a chat/vision model id instead of an embedding model, unsupported param leaked into the main dict (the comment notes they should be moved to extra_body), bad/empty inputs list, expired NVIDIA key.

Related errors

AI-assisted analysis of microsoft/semantic-kernel@c028a0c7dc (2026-08-13). Data as JSON: /api/errors/85d45c633e728b6e. Report an issue: GitHub.