microsoft/autogen · error · ArgumentException

Unknown message type: {m.GetType()}

Error message

Unknown message type: {m.GetType()}

What it means

While flattening a MultiModalMessage into ChatMessageContentItem parts, this ArgumentException fires when an inner message of the multi-modal collection is neither TextMessage nor ImageMessage. Only text and image parts can map to OpenAI content items.

Source

Thrown at dotnet/src/AutoGen.OpenAI.V1/Extension/MessageExtension.cs:108

            }
            else if (message is ToolCallResultMessage toolCallResult)
            {
                return toolCallResult.ToolCalls.Select(m =>
                {
                    var msg = new ChatRequestToolMessage(m.Result, m.FunctionName);

                    return msg;
                });
            }
            else if (message is MultiModalMessage multiModalMessage)
            {
                var messageContent = multiModalMessage.Content.Select<IMessage, ChatMessageContentItem>(m =>
                {
                    return m switch
                    {
                        TextMessage textMessage => new ChatMessageTextContentItem(textMessage.Content),
                        ImageMessage imageMessage => new ChatMessageImageContentItem(new Uri(imageMessage.Url ?? imageMessage.BuildDataUri())),
                        _ => throw new ArgumentException($"Unknown message type: {m.GetType()}")
                    };
                });

                var msg = new ChatRequestUserMessage(messageContent);
                return [msg];
            }
            else if (message is AggregateMessage<ToolCallMessage, ToolCallResultMessage> aggregateMessage)
            {
                // convert as user message
                var resultMessage = aggregateMessage.Message2;
                return resultMessage.ToolCalls.Select(m => new ChatRequestUserMessage(m.Result));
            }
            else if (message is Message msg)
            {
                if (msg.Role == Role.System)
                {
                    var systemMessage = new ChatRequestSystemMessage(msg.Content ?? string.Empty);
                    return [systemMessage];

View on GitHub (pinned to 027ecf0a37)

Solutions

1. Restrict MultiModalMessage.Content to TextMessage and ImageMessage instances only
2. Flatten nested structures (e.g. inner MultiModalMessage) into their text/image parts before building the outer message
3. Add an assert/validate step when composing multi-modal content dynamically

Example fix

// before
var mm = new MultiModalMessage(Role.User,
    new IMessage[] { textMsg, imageMsg, otherAgentToolCall });

// after
var mm = new MultiModalMessage(Role.User,
    new IMessage[] { textMsg, imageMsg });
// convert otherAgentToolCall to a TextMessage if its content matters

bool partsValid = multiModalMessage.Content.All(m => m is TextMessage or ImageMessage);

static bool IsSupportedContentPart(IMessage m) => m is TextMessage or ImageMessage;

catch (ArgumentException ex) when (ex.Message.Contains("Unknown message type"))
{
    var parts = mm.Content.OfType<TextMessage>().Select(t => t.Content);
    return string.Join("\n", parts);
}

When it happens

Trigger: Constructing MultiModalMessage.Content with a ToolCallMessage, another MultiModalMessage, or any custom IMessage implementation instead of TextMessage/ImageMessage elements.

Common situations: Programmatically aggregating mixed conversation turns into one multi-modal message; custom IMessage types introduced by user middleware leaking into multi-modal collections.

Related errors

• Unsupported config type {config.GetType()}
• Messages should not be provided in options
• Invalid message type
• Parameter name cannot be null
• Content is null and metadata is null

AI-assisted analysis of microsoft/autogen@027ecf0a37 (2026-08-15). Data as JSON: /api/errors/80606658a493683c. Report an issue: GitHub.