{"record":{"id":"f6f3b43e810df825","repo":"microsoft/autogen","slug":"multimodalmessage-is-not-supported-when-message-fr-f6f3b4","errorCode":null,"errorMessage":"MultiModalMessage is not supported when message.From is the same with agent","messagePattern":"MultiModalMessage is not supported when message\\.From is the same with agent","errorType":"exception","errorClass":"ArgumentException","httpStatus":null,"severity":"error","filePath":"dotnet/src/AutoGen.OpenAI/Middleware/OpenAIChatRequestMessageConnector.cs","lineNumber":282,"sourceCode":"\n    private IEnumerable<ChatMessage> ProcessImageMessage(IAgent agent, ImageMessage message)\n    {\n        if (agent.Name == message.From)\n        {\n            // image message from assistant is not supported\n            throw new ArgumentException(\"ImageMessage is not supported when message.From is the same with agent\");\n        }\n\n        var imageContentItem = this.CreateChatMessageImageContentItemFromImageMessage(message);\n        return [new UserChatMessage([imageContentItem]) { ParticipantName = message.From }];\n    }\n\n    private IEnumerable<ChatMessage> ProcessMultiModalMessage(IAgent agent, MultiModalMessage message)\n    {\n        if (agent.Name == message.From)\n        {\n            // image message from assistant is not supported\n            throw new ArgumentException(\"MultiModalMessage is not supported when message.From is the same with agent\");\n        }\n\n        IEnumerable<ChatMessageContentPart> items = message.Content.Select<IMessage, ChatMessageContentPart>(ci => ci switch\n        {\n            TextMessage text => ChatMessageContentPart.CreateTextPart(text.Content),\n            ImageMessage image => this.CreateChatMessageImageContentItemFromImageMessage(image),\n            _ => throw new NotImplementedException(),\n        });\n\n        return [new UserChatMessage(items) { ParticipantName = message.From }];\n    }\n\n    private ChatMessageContentPart CreateChatMessageImageContentItemFromImageMessage(ImageMessage message)\n    {\n        return message.Data is null && message.Url is not null\n            ? ChatMessageContentPart.CreateImagePart(new Uri(message.Url))\n            : ChatMessageContentPart.CreateImagePart(message.Data, message.Data?.MediaType);\n    }","sourceCodeStart":264,"sourceCodeEnd":300,"githubUrl":"https://github.com/microsoft/autogen/blob/027ecf0a379bcc1d09956d46d12d44a3ad9cee14/dotnet/src/AutoGen.OpenAI/Middleware/OpenAIChatRequestMessageConnector.cs#L264-L300","documentation":"ProcessMultiModalMessage throws when a MultiModalMessage's From equals the agent's name. Just like ImageMessage, the OpenAI chat API only accepts multimodal (image+text) parts in user messages, so an assistant-authored multimodal message cannot be replayed to its own author.","triggerScenarios":"A group chat or handcrafted history sends a MultiModalMessage with from set to the receiving agent's name.","commonSituations":"Multi-agent workflows that broadcast all messages to every participant; storing and replaying full conversation logs including assistant multimodal turns.","solutions":["Exclude self-authored MultiModalMessages from the message list passed to the agent.","Set From to the actual requester (or null) when the multimodal content is genuine user input.","Convert the multimodal content to a plain TextMessage (describe images as text) if it must appear in the assistant's own history."],"exampleFix":"// before\nvar msg = new MultiModalMessage(Role.Assistant, items, from: agent.Name);\nawait agent.SendAsync(new[] { msg });\n\n// after\nvar msg = new MultiModalMessage(Role.User, items, from: \"user-1\");\nawait agent.SendAsync(new[] { msg });","handlingStrategy":"validation","validationCode":"var safeHistory = messages.Where(m => m is not MultiModalMessage mm || mm.From != agent.Name).ToList();\nawait agent.SendAsync(safeHistory);","typeGuard":"static bool IsReplayableToAgent(IMessage m, string agentName) => m switch\n{\n    MultiModalMessage mm => mm.From != agentName,\n    _ => true,\n};","tryCatchPattern":"catch (ArgumentException ex) when (ex.Message.Contains(\"MultiModalMessage is not supported\"))\n{\n    var filtered = messages.Where(m => m is not MultiModalMessage mm || mm.From != agent.Name);\n    return await agent.SendAsync(filtered);\n}","preventionTips":["Treat assistant-authored multimodal content as non-replayable in group-chat design.","Document per-connector replay rules next to your group chat orchestration code."],"tags":["openai","message-connector","multimodal","group-chat","autogen"],"backgroundTag":null,"analyzedSha":"027ecf0a379bcc1d09956d46d12d44a3ad9cee14","analyzedAt":"2026-08-15T03:38:00.719Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}