microsoft/autogen · error · InvalidOperationException

MultiModalMessage is not supported in the semantic kernel if

Error message

MultiModalMessage is not supported in the semantic kernel if it's from self.

What it means

ProcessMessageForSelf(MultiModalMessage) throws unconditionally: Semantic Kernel's chat content model cannot represent a multimodal message authored by the assistant (self), because SK restricts image parts to user-role content.

Source

Thrown at dotnet/src/AutoGen.SemanticKernel/Middleware/SemanticKernelChatMessageContentConnector.cs:202

        if (message.Url is not null)
        {
            collectionItems.Add(new ImageContent(new Uri(message.Url)));
        }
        else if (message.BuildDataUri() is string dataUri)
        {
            collectionItems.Add(new ImageContent(dataUri));
        }
        else
        {
            throw new InvalidOperationException("ImageMessage must have Url or DataUri");
        }

        return [new ChatMessageContent(AuthorRole.User, collectionItems)];
    }

    private IEnumerable<ChatMessageContent> ProcessMessageForSelf(MultiModalMessage message)
    {
        throw new System.InvalidOperationException("MultiModalMessage is not supported in the semantic kernel if it's from self.");
    }

    private IEnumerable<ChatMessageContent> ProcessMessageForOthers(MultiModalMessage message)
    {
        var collections = new ChatMessageContentItemCollection();
        foreach (var item in message.Content)
        {
            if (item is TextMessage textContent)
            {
                collections.Add(new TextContent(textContent.Content));
            }
            else if (item is ImageMessage imageContent)
            {
                collections.Add(new ImageContent(new Uri(imageContent.Url ?? imageContent.BuildDataUri())));
            }
            else
            {
                throw new InvalidOperationException($"Unsupported message type: {item.GetType().Name}");

View on GitHub (pinned to 027ecf0a37)

Solutions

Filter self-authored MultiModalMessages out of the history for SK agents.
Downgrade self multimodal content to a TextMessage (keep only the text items) before replay.
Re-author the message with From set to the original user if it must be seen as input.

Example fix

// before
var history = groupChatMessages; // includes skAgent's own MultiModalMessage
await skAgent.SendAsync(history);

// after
var history = groupChatMessages.Where(m => m is not MultiModalMessage mm || mm.From != skAgent.Name);
await skAgent.SendAsync(history);

Defensive patterns

Strategy: validation

Validate before calling

var safeForSk = messages.Where(m => m is not MultiModalMessage mm || mm.From != skAgent.Name).ToList();
await skAgent.SendAsync(safeForSk);

Type guard

static bool IsSkReplayable(IMessage m, string agentName) => m switch
{
    MultiModalMessage mm => mm.From != agentName,
    _ => true,
};

Try / catch

catch (InvalidOperationException ex) when (ex.Message.Contains("from self"))
{
    var filtered = messages.Where(m => m is not MultiModalMessage mm || mm.From != skAgent.Name);
    return await skAgent.SendAsync(filtered);
}

Prevention

Never broadcast assistant multimodal replies back to their own author in SK pipelines.
Persist assistant multimodal turns as text summaries if they must appear in later context.

When it happens

Trigger: A MultiModalMessage whose From equals the SK agent's name is replayed to that agent (group-chat broadcast, history restore).

Common situations: Round-robin group chats that send all history to every agent; saving assistant multimodal replies and feeding them back.

Related errors

AI-assisted analysis of microsoft/autogen@027ecf0a37 (2026-08-15). Data as JSON: /api/errors/349663d70f3ffbe1. Report an issue: GitHub.