microsoft/autogen · error · NotImplementedException

Unsupported reply content type

Error message

Unsupported reply content type

What it means

OpenAIChatCompletionService.GetChatCompletionAsync maps the agent's reply into an OpenAI completion response only when reply.GetContent() is a string. Any other content (image bytes, multimodal collections, structured objects) hits NotImplementedException 'Unsupported reply content type' — this endpoint is text-only on the response side.

Source

Thrown at dotnet/src/AutoGen.WebAPI/OpenAI/Service/OpenAIChatCompletionService.cs:54

        {
            var message = new OpenAIChatCompletionMessage()
            {
                Content = content,
            };

            var choice = new OpenAIChatCompletionChoice()
            {
                Message = message,
                Index = 0,
                FinishReason = "stop",
            };

            openAIChatCompletion.Choices = [choice];

            return openAIChatCompletion;
        }

        throw new NotImplementedException("Unsupported reply content type");
    }

    public async IAsyncEnumerable<OpenAIChatCompletion> GetStreamingChatCompletionAsync(OpenAIChatCompletionOption request)
    {
        if (this.agent is IStreamingAgent streamingAgent)
        {
            var messages = this.ProcessMessages(request.Messages ?? Array.Empty<OpenAIMessage>());

            var generateOption = this.ProcessReplyOptions(request);

            await foreach (var reply in streamingAgent.GenerateStreamingReplyAsync(messages, generateOption))
            {
                var openAIChatCompletion = new OpenAIChatCompletion()
                {
                    Created = DateTimeOffset.UtcNow.Ticks / TimeSpan.TicksPerMillisecond / 1000,
                    Model = this.agent.Name,
                };

View on GitHub (pinned to 027ecf0a37)

Solutions

1. Ensure the registered agent replies with string content (TextMessage / MessageEnvelope<string>)
2. In the agent, stringify non-text results (URL for images, JSON for structured data) before returning
3. Do not expose image/multimodal-output agents through this completion endpoint; serve them from a dedicated route

Example fix

// before
return new ImageMessage(Role.Assistant, imageUri, from: Name); // GetContent() != string -> throws

// after
return new TextMessage(Role.Assistant, imageUri.ToString(), from: Name);

var content = reply.GetContent();
if (content is not string)
    throw new InvalidOperationException($"Agent '{agent.Name}' must reply with string content for the OpenAI endpoint (got {content?.GetType().Name ?? "null"}).");

static bool HasStringContent(IMessage reply) => reply.GetContent() is string;

try { return await service.GetChatCompletionAsync(request); }
catch (NotImplementedException ex) when (ex.Message.Contains("reply content"))
{
    return StatusCode(501, "Backing agent does not produce text replies through this endpoint.");
}

When it happens

Trigger: The backing agent returns an ImageMessage, MultiModalMessage, or any reply whose GetContent() is not a string (e.g. null content or byte[]/Uri content).

Common situations: Registering an image-generation or multimodal agent behind the OpenAI-compatible WebAPI; an agent returning an empty content message; tool-call replies whose content is an object.

Related errors

• Exception of type 'System.Text.Json.JsonException' was throw
• Unsupported message type {m.GetType()}
• Value cannot be null. (Parameter 'Content')
• Unsupported content type {item.GetType()}
• Please set OPENAI_API_KEY environment variable.

AI-assisted analysis of microsoft/autogen@027ecf0a37 (2026-08-15). Data as JSON: /api/errors/ea0aa7c63ca926dd. Report an issue: GitHub.