microsoft/autogen · error · ArgumentException

MultiModalMessage is not supported when message.From is the

Error message

MultiModalMessage is not supported when message.From is the same with agent

What it means

ArgumentException thrown by ProcessMultiModalMessage when a MultiModalMessage's From equals the agent's name. Like single images, mixed text+image content is only representable as a ChatRequestUserMessage, so an assistant-authored multimodal message cannot be fed back to the same agent.

Source

Thrown at dotnet/src/AutoGen.OpenAI.V1/Middleware/OpenAIChatRequestMessageConnector.cs:309

    private IEnumerable<ChatRequestMessage> ProcessImageMessage(IAgent agent, ImageMessage message)
    {
        if (agent.Name == message.From)
        {
            // image message from assistant is not supported
            throw new ArgumentException("ImageMessage is not supported when message.From is the same with agent");
        }

        var imageContentItem = this.CreateChatMessageImageContentItemFromImageMessage(message);
        return [new ChatRequestUserMessage([imageContentItem]) { Name = message.From }];
    }

    private IEnumerable<ChatRequestMessage> ProcessMultiModalMessage(IAgent agent, MultiModalMessage message)
    {
        if (agent.Name == message.From)
        {
            // image message from assistant is not supported
            throw new ArgumentException("MultiModalMessage is not supported when message.From is the same with agent");
        }

        IEnumerable<ChatMessageContentItem> items = message.Content.Select<IMessage, ChatMessageContentItem>(ci => ci switch
        {
            TextMessage text => new ChatMessageTextContentItem(text.Content),
            ImageMessage image => this.CreateChatMessageImageContentItemFromImageMessage(image),
            _ => throw new NotImplementedException(),
        });

        return [new ChatRequestUserMessage(items) { Name = message.From }];
    }

    private ChatMessageImageContentItem CreateChatMessageImageContentItemFromImageMessage(ImageMessage message)
    {
        return message.Data is null && message.Url is not null
            ? new ChatMessageImageContentItem(new Uri(message.Url))
            : new ChatMessageImageContentItem(message.Data, message.Data?.MediaType);
    }

View on GitHub (pinned to 027ecf0a37)

Solutions

Filter out the agent's own MultiModalMessage instances before sending history to that agent
Store assistant multimodal output as text-only messages in shared history
Send the multimodal content to a different consumer agent

Example fix

// before
var history = new List<IMessage> { multiModalReplyFromSelf };
await agent.SendAsync(history); // throws

// after
var history = allMessages.Where(m => m.From != agent.Name || m is TextMessage);
await agent.SendAsync(history);

Defensive patterns

Strategy: validation

Validate before calling

var safeHistory = messages.Where(m => m is not MultiModalMessage mm || mm.From != agent.Name);

Type guard

static bool IsReplayableToAgent(MultiModalMessage m, IAgent agent) => m.From != agent.Name;

Try / catch

catch (ArgumentException ex) when (ex.Message.Contains("MultiModalMessage is not supported")) { /* drop self multimodal from history and resend */ }

Prevention

Exclude the originator's multimodal messages when replaying
Convert assistant multimodal output to text summaries in group chat memory

When it happens

Trigger: A MultiModalMessage with From == agent.Name reaching this connector — e.g. a DALL-E/vision agent's multimodal reply looped back into its own context in a group chat.

Common situations: Group chat histories broadcast to all participants including the originator; multimodal agent output cached and re-sent on the next turn.

Related errors

AI-assisted analysis of microsoft/autogen@027ecf0a37 (2026-08-15). Data as JSON: /api/errors/eb2b8c3f29802e00. Report an issue: GitHub.