microsoft/semantic-kernel · error · KernelException

Unexpected response from model

Error message

Unexpected response from model

What it means

Thrown by GetStreamingChatMessageContentFromStreamResponse when a streamed chat completion chunk has no choices (response.Choices is null or empty, so FirstOrDefault yields null). The streaming chat path cannot build a StreamingChatMessageContent without a delta, so it raises a KernelException with the raw response in Data["ResponseData"].

Source

Thrown at dotnet/src/Connectors/Connectors.HuggingFace/Core/HuggingFaceMessageApiClient.cs:261

                SystemFingerPrint = response.SystemFingerprint,
                Created = response.Created,
                FinishReason = choice.FinishReason,
                LogProbs = choice.LogProbs,
            };

            var streamChat = new StreamingChatMessageContent(
                choice.Delta?.Role is not null ? new AuthorRole(choice.Delta.Role) : null,
                choice.Delta?.Content,
                response,
                choice.Index,
                modelId,
                Encoding.UTF8,
                metadata);

            return streamChat;
        }

        throw new KernelException("Unexpected response from model")
        {
            Data = { { "ResponseData", response } },
        };
    }

    private async IAsyncEnumerable<StreamingChatMessageContent> ProcessChatResponseStreamAsync(Stream stream, string? modelId, [EnumeratorCancellation] CancellationToken cancellationToken)
    {
        await foreach (var content in this.ParseChatResponseStreamAsync(stream, cancellationToken).ConfigureAwait(false))
        {
            yield return GetStreamingChatMessageContentFromStreamResponse(content, modelId);
        }
    }

    private ChatCompletionRequest CreateChatRequest(
        ChatHistory chatHistory,
        HuggingFacePromptExecutionSettings huggingFaceExecutionSettings,
        string? modelId)
    {

View on GitHub (pinned to c028a0c7dc)

Solutions

1. Inspect KernelException.Data["ResponseData"] to see the offending chunk and confirm whether it is a benign framing event.
2. Upgrade the connector to a version compatible with the HF TGI streaming schema you are hitting.
3. If reproducible only for certain models, test with a different model to isolate a server-side behavior difference.
4. Retry once; transient empty chunks may not recur.

Example fix

// before
await foreach (var c in svc.GetStreamingTextContentsAsync(prompt, settings)) { /* throws mid-stream */ }

// after (isolate offending chunk)
try { await foreach (var c in svc.GetStreamingTextContentsAsync(prompt, settings)) yield return c; }
catch (KernelException ex)
{
    var raw = ex.Data.Contains("ResponseData") ? ex.Data["ResponseData"] : null;
    logger.LogError("Empty-choice stream chunk: {Chunk}", raw);
    throw;
}

try { await foreach (var c in svc.GetStreamingTextContentsAsync(prompt, settings)) yield return c; }
catch (KernelException ex)
{
    var raw = ex.Data.Contains("ResponseData") ? ex.Data["ResponseData"]?.ToString() : "<none>";
    logger.LogError("Empty-choice stream chunk: {Chunk}", raw);
}

When it happens

Trigger: HuggingFace streaming endpoint emits a chunk whose 'choices' array is empty or absent (e.g. the first/last framing chunks, usage-only chunks, or error frames), and the connector hits this branch instead of skipping.

Common situations: Models that emit empty-choice framing chunks; endpoint/schema changes adding non-choice events; mismatched streaming protocol between connector version and HF Text Generation Inference server.

Related errors

• Unexpected response from model
• dictionary
• Agent Failure - Run not created for thread: ${threadId}
• The streaming configuration must be null for non-streaming r
• The streaming configuration must have StreamFinalResponse se

AI-assisted analysis of microsoft/semantic-kernel@c028a0c7dc (2026-08-13). Data as JSON: /api/errors/a0b2d266734acc85. Report an issue: GitHub.