microsoft/autogen · error · ValueError

Unknown content part: {part}

Error message

Unknown content part: {part}

What it means

When converting an autogen multimodal UserMessage to OpenAI's content-part format, _message_transform.py accepts only str and Image parts. Any other object type in message.content (which is typed List[str | Image] but Python does not enforce it at runtime) raises ValueError('Unknown content part: {part}'). The transformer deliberately rejects unknown part types rather than guessing a serialization.

Source

Thrown at python/packages/autogen-ext/src/autogen_ext/models/openai/_message_transform.py:239

def _set_multimodal_content(
    message: LLMMessage, context: Dict[str, Any]
) -> Dict[str, List[ChatCompletionContentPartParam]]:
    assert isinstance(message, (UserMessage, AssistantMessage))
    prepend = context.get("prepend_name", False)
    parts: List[ChatCompletionContentPartParam] = []

    for idx, part in enumerate(message.content):
        if isinstance(part, str):
            # If prepend, Append the name to the first text part
            text = f"{message.source} said:\n" + part if prepend and idx == 0 else part
            parts.append(ChatCompletionContentPartTextParam(type="text", text=text))
        elif isinstance(part, Image):
            # TODO: support url based images
            # TODO: support specifying details
            parts.append(cast(ChatCompletionContentPartImageParam, part.to_openai_format()))
        else:
            raise ValueError(f"Unknown content part: {part}")

    return {"content": parts}


def _set_tool_calls(
    message: LLMMessage, context: Dict[str, Any]
) -> Dict[str, List[ChatCompletionMessageToolCallParam]]:
    assert isinstance(message.content, list)
    assert isinstance(message, AssistantMessage)
    return {
        "tool_calls": [func_call_to_oai(x) for x in message.content],
    }


def _set_thought_as_content(message: LLMMessage, context: Dict[str, Any]) -> Dict[str, str | None]:
    assert isinstance(message, AssistantMessage)
    return {"content": message.thought}

View on GitHub (pinned to 027ecf0a37)

Solutions

1. Use only str and autogen_core.models.Image parts in UserMessage.content lists
2. Wrap non-string data explicitly: load images via Image.from_file()/Image.from_base64() instead of passing PIL objects or dicts
3. Align autogen-core and autogen-ext versions (pip install -U autogen-core autogen-ext) so content types and transformers agree

Example fix

# before
UserMessage(source='user', content=['look', {'type':'image_url','image_url':{'url':'data:image/png;base64,...'}}])
# ValueError: Unknown content part: {...}

# after
from autogen_core.models import UserMessage, Image
UserMessage(source='user', content=['look', Image.from_base64('image/png', base64_data)])

from autogen_core.models import UserMessage, Image

def content_parts_valid(msg) -> bool:
    if not (isinstance(msg, UserMessage) and isinstance(msg.content, list)):
        return True
    return all(isinstance(p, (str, Image)) for p in msg.content)

assert all(content_parts_valid(m) for m in messages)

from typing import Any

def is_valid_content_part(part: Any) -> bool:
    return isinstance(part, (str, Image))

try:
    result = await client.create(messages)
except ValueError as e:
    if 'Unknown content part' in str(e):
        messages = [normalize_parts(m) for m in messages]  # map dicts/PIL -> str|Image
        result = await client.create(messages)
    else:
        raise

When it happens

Trigger: Passing a UserMessage whose content list contains anything besides str or Image instances — e.g. a dict {'type':'text','text':'hi'}, a MultiModalContent object from a newer autogen version, a PIL image, or a custom class. Reached via create()/create_stream() on OpenAI-compatible clients that use the shared transformer.

Common situations: Mixing autogen-core versions where a newer content type (e.g. MultiModalContent) is produced by one package but an older autogen-ext transformer doesn't know it; hand-building message content with raw OpenAI dicts instead of autogen types; putting a URL string/object where an Image is expected.

Related errors

• Invalid aggregate message {reason}
• Only one choice is supported in streaming response
• Parameter name cannot be null
• Value cannot be null. (Parameter 'functionContract.Name')
• MultiModalMessage is not supported when message.From is the

AI-assisted analysis of microsoft/autogen@027ecf0a37 (2026-08-15). Data as JSON: /api/errors/df5970d2e1c7065e. Report an issue: GitHub.