{"record":{"id":"349663d70f3ffbe1","repo":"microsoft/autogen","slug":"multimodalmessage-is-not-supported-in-the-semantic","errorCode":null,"errorMessage":"MultiModalMessage is not supported in the semantic kernel if it's from self.","messagePattern":"MultiModalMessage is not supported in the semantic kernel if it's from self\\.","errorType":"exception","errorClass":"InvalidOperationException","httpStatus":null,"severity":"error","filePath":"dotnet/src/AutoGen.SemanticKernel/Middleware/SemanticKernelChatMessageContentConnector.cs","lineNumber":202,"sourceCode":"        if (message.Url is not null)\n        {\n            collectionItems.Add(new ImageContent(new Uri(message.Url)));\n        }\n        else if (message.BuildDataUri() is string dataUri)\n        {\n            collectionItems.Add(new ImageContent(dataUri));\n        }\n        else\n        {\n            throw new InvalidOperationException(\"ImageMessage must have Url or DataUri\");\n        }\n\n        return [new ChatMessageContent(AuthorRole.User, collectionItems)];\n    }\n\n    private IEnumerable<ChatMessageContent> ProcessMessageForSelf(MultiModalMessage message)\n    {\n        throw new System.InvalidOperationException(\"MultiModalMessage is not supported in the semantic kernel if it's from self.\");\n    }\n\n    private IEnumerable<ChatMessageContent> ProcessMessageForOthers(MultiModalMessage message)\n    {\n        var collections = new ChatMessageContentItemCollection();\n        foreach (var item in message.Content)\n        {\n            if (item is TextMessage textContent)\n            {\n                collections.Add(new TextContent(textContent.Content));\n            }\n            else if (item is ImageMessage imageContent)\n            {\n                collections.Add(new ImageContent(new Uri(imageContent.Url ?? imageContent.BuildDataUri())));\n            }\n            else\n            {\n                throw new InvalidOperationException($\"Unsupported message type: {item.GetType().Name}\");","sourceCodeStart":184,"sourceCodeEnd":220,"githubUrl":"https://github.com/microsoft/autogen/blob/027ecf0a379bcc1d09956d46d12d44a3ad9cee14/dotnet/src/AutoGen.SemanticKernel/Middleware/SemanticKernelChatMessageContentConnector.cs#L184-L220","documentation":"ProcessMessageForSelf(MultiModalMessage) throws unconditionally: Semantic Kernel's chat content model cannot represent a multimodal message authored by the assistant (self), because SK restricts image parts to user-role content.","triggerScenarios":"A MultiModalMessage whose From equals the SK agent's name is replayed to that agent (group-chat broadcast, history restore).","commonSituations":"Round-robin group chats that send all history to every agent; saving assistant multimodal replies and feeding them back.","solutions":["Filter self-authored MultiModalMessages out of the history for SK agents.","Downgrade self multimodal content to a TextMessage (keep only the text items) before replay.","Re-author the message with From set to the original user if it must be seen as input."],"exampleFix":"// before\nvar history = groupChatMessages; // includes skAgent's own MultiModalMessage\nawait skAgent.SendAsync(history);\n\n// after\nvar history = groupChatMessages.Where(m => m is not MultiModalMessage mm || mm.From != skAgent.Name);\nawait skAgent.SendAsync(history);","handlingStrategy":"validation","validationCode":"var safeForSk = messages.Where(m => m is not MultiModalMessage mm || mm.From != skAgent.Name).ToList();\nawait skAgent.SendAsync(safeForSk);","typeGuard":"static bool IsSkReplayable(IMessage m, string agentName) => m switch\n{\n    MultiModalMessage mm => mm.From != agentName,\n    _ => true,\n};","tryCatchPattern":"catch (InvalidOperationException ex) when (ex.Message.Contains(\"from self\"))\n{\n    var filtered = messages.Where(m => m is not MultiModalMessage mm || mm.From != skAgent.Name);\n    return await skAgent.SendAsync(filtered);\n}","preventionTips":["Never broadcast assistant multimodal replies back to their own author in SK pipelines.","Persist assistant multimodal turns as text summaries if they must appear in later context."],"tags":["semantic-kernel","multimodal","group-chat","autogen","csharp"],"backgroundTag":null,"analyzedSha":"027ecf0a379bcc1d09956d46d12d44a3ad9cee14","analyzedAt":"2026-08-15T03:38:00.719Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}