microsoft/autogen · error · ValueError

Model does not support vision and image was provided

Error message

Model does not support vision and image was provided

What it means

Thrown by OpenAIChatCompletionClient when a message contains an Image part but the client's model_info declares vision support as False. The client validates capability metadata before sending the request, so this fails client-side before any API call. It exists to prevent sending image content to text-only models, which would otherwise fail opaquely at the API.

Source

Thrown at python/packages/autogen-ext/src/autogen_ext/models/openai/_openai_client.py:577

        if response_format_value is not None and "response_format" in create_args:
            warnings.warn(
                "response_format is found in extra_create_args while json_output is set to a Pydantic model class. "
                "Skipping the response_format in extra_create_args in favor of the json_output. "
                "Structured output will be used.",
                UserWarning,
                stacklevel=2,
            )
            # If using beta client, remove response_format from create_args to prevent passing it twice
            del create_args["response_format"]

        # TODO: allow custom handling.
        # For now we raise an error if images are present and vision is not supported
        if self.model_info["vision"] is False:
            for message in messages:
                if isinstance(message, UserMessage):
                    if isinstance(message.content, list) and any(isinstance(x, Image) for x in message.content):
                        raise ValueError("Model does not support vision and image was provided")

        if self.model_info["json_output"] is False and json_output is True:
            raise ValueError("Model does not support JSON output.")

        if not self.model_info.get("multiple_system_messages", False):
            # Some models accept only one system message(or, it will read only the last one)
            # So, merge system messages into one (if multiple and continuous)
            system_message_content = ""
            _messages: List[LLMMessage] = []
            _first_system_message_idx = -1
            _last_system_message_idx = -1
            # Index of the first system message for adding the merged system message at the correct position
            for idx, message in enumerate(messages):
                if isinstance(message, SystemMessage):
                    if _first_system_message_idx == -1:
                        _first_system_message_idx = idx
                    elif _last_system_message_idx + 1 != idx:
                        # That case, system message is not continuous

View on GitHub (pinned to 027ecf0a37)

Solutions

  1. Switch to a vision-capable model (e.g. gpt-4o) whose model_info has vision=True
  2. If the model actually supports vision, pass a corrected model_info dict to the client constructor with 'vision': True
  3. Remove Image parts from the UserMessage content before calling create
  4. If using an older model entry, regenerate model_info via the model info lookup helpers instead of a stale hardcoded dict

Example fix

// before
client = OpenAIChatCompletionClient(model="gpt-3.5-turbo")
msg = UserMessage(content=["describe", Image.from_file("cat.png")], source="user")
await client.create([msg])

// after
client = OpenAIChatCompletionClient(model="gpt-4o")  # vision-capable
msg = UserMessage(content=["describe", Image.from_file("cat.png")], source="user")
await client.create([msg])
Defensive patterns

Strategy: validation

Validate before calling

from autogen_core import Image
from autogen_agentchat.messages import UserMessage

def has_image(messages) -> bool:
    return any(
        isinstance(m, UserMessage) and isinstance(m.content, list)
        and any(isinstance(p, Image) for p in m.content)
        for m in messages
    )

if has_image(messages) and not client.info.get("vision", False):
    raise RuntimeError("switch to a vision-capable model or strip images")

Type guard

def is_vision_message_list(messages: Sequence[LLMMessage], vision_supported: bool) -> bool:
    """True when it is safe to send these messages: no images, or vision supported."""
    return vision_supported or not has_image(messages)

Try / catch

try:
    result = await client.create(messages)
except ValueError as e:
    if "does not support vision" in str(e):
        messages = [strip_images(m) for m in messages]  # degrade to text
        result = await client.create(messages)
    else:
        raise

Prevention

When it happens

Trigger: Calling create/create_stream with a UserMessage whose content is a list containing one or more Image objects, while model_info['vision'] is False (e.g. a text-only model like gpt-3.5-turbo or a custom model_info with vision=False). Only UserMessage instances are checked; SystemMessage/AssistantMessage content is not scanned.

Common situations: Using a custom/open-hosted model with a hand-written model_info dict that omits or sets vision=False; upgrading autogen where older model_info entries lacked the vision key; replaying a vision conversation against a text-only model in tests.

Related errors

AI-assisted analysis of microsoft/autogen@027ecf0a37 (2026-08-15). Data as JSON: /api/errors/95873dcdd280b708. Report an issue: GitHub.