microsoft/autogen · error · ArgumentException

MultiModalMessage is not supported when message.From is the

Error message

MultiModalMessage is not supported when message.From is the same with agent

What it means

ProcessMultiModalMessage throws when a MultiModalMessage's From equals the agent's name. Just like ImageMessage, the OpenAI chat API only accepts multimodal (image+text) parts in user messages, so an assistant-authored multimodal message cannot be replayed to its own author.

Source

Thrown at dotnet/src/AutoGen.OpenAI/Middleware/OpenAIChatRequestMessageConnector.cs:282

    private IEnumerable<ChatMessage> ProcessImageMessage(IAgent agent, ImageMessage message)
    {
        if (agent.Name == message.From)
        {
            // image message from assistant is not supported
            throw new ArgumentException("ImageMessage is not supported when message.From is the same with agent");
        }

        var imageContentItem = this.CreateChatMessageImageContentItemFromImageMessage(message);
        return [new UserChatMessage([imageContentItem]) { ParticipantName = message.From }];
    }

    private IEnumerable<ChatMessage> ProcessMultiModalMessage(IAgent agent, MultiModalMessage message)
    {
        if (agent.Name == message.From)
        {
            // image message from assistant is not supported
            throw new ArgumentException("MultiModalMessage is not supported when message.From is the same with agent");
        }

        IEnumerable<ChatMessageContentPart> items = message.Content.Select<IMessage, ChatMessageContentPart>(ci => ci switch
        {
            TextMessage text => ChatMessageContentPart.CreateTextPart(text.Content),
            ImageMessage image => this.CreateChatMessageImageContentItemFromImageMessage(image),
            _ => throw new NotImplementedException(),
        });

        return [new UserChatMessage(items) { ParticipantName = message.From }];
    }

    private ChatMessageContentPart CreateChatMessageImageContentItemFromImageMessage(ImageMessage message)
    {
        return message.Data is null && message.Url is not null
            ? ChatMessageContentPart.CreateImagePart(new Uri(message.Url))
            : ChatMessageContentPart.CreateImagePart(message.Data, message.Data?.MediaType);
    }

View on GitHub (pinned to 027ecf0a37)

Solutions

  1. Exclude self-authored MultiModalMessages from the message list passed to the agent.
  2. Set From to the actual requester (or null) when the multimodal content is genuine user input.
  3. Convert the multimodal content to a plain TextMessage (describe images as text) if it must appear in the assistant's own history.

Example fix

// before
var msg = new MultiModalMessage(Role.Assistant, items, from: agent.Name);
await agent.SendAsync(new[] { msg });

// after
var msg = new MultiModalMessage(Role.User, items, from: "user-1");
await agent.SendAsync(new[] { msg });
Defensive patterns

Strategy: validation

Validate before calling

var safeHistory = messages.Where(m => m is not MultiModalMessage mm || mm.From != agent.Name).ToList();
await agent.SendAsync(safeHistory);

Type guard

static bool IsReplayableToAgent(IMessage m, string agentName) => m switch
{
    MultiModalMessage mm => mm.From != agentName,
    _ => true,
};

Try / catch

catch (ArgumentException ex) when (ex.Message.Contains("MultiModalMessage is not supported"))
{
    var filtered = messages.Where(m => m is not MultiModalMessage mm || mm.From != agent.Name);
    return await agent.SendAsync(filtered);
}

Prevention

When it happens

Trigger: A group chat or handcrafted history sends a MultiModalMessage with from set to the receiving agent's name.

Common situations: Multi-agent workflows that broadcast all messages to every participant; storing and replaying full conversation logs including assistant multimodal turns.

Related errors

AI-assisted analysis of microsoft/autogen@027ecf0a37 (2026-08-15). Data as JSON: /api/errors/f6f3b43e810df825. Report an issue: GitHub.