{"record":{"id":"b7fd5b8f281e27bd","repo":"microsoft/autogen","slug":"model-model-family-does-not-support-vision","errorCode":null,"errorMessage":"model {model_family} does not support vision.","messagePattern":"model (.+?) does not support vision\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/packages/autogen-ext/src/autogen_ext/tools/mcp/_host/_sampling.py","lineNumber":48,"sourceCode":"    Handles text and image content conversion, with vision model validation for images.\n\n    Args:\n        content: MCP content object (text, image, or audio)\n        model_info: Optional model information for vision capability checking\n\n    Returns:\n        Converted content as string or Image object\n\n    Raises:\n        RuntimeError: If image content is provided but model doesn't support vision\n        ValueError: If content type is unsupported\n    \"\"\"\n    if content.type == \"text\":\n        return content.text\n    elif content.type == \"image\":\n        if model_info and not model_info.get(\"vision\", False):\n            model_family = model_info.get(\"family\", \"unknown\")\n            raise RuntimeError(f\"model {model_family} does not support vision.\")\n\n        # Decode base64 image data and create PIL Image\n        image_data = base64.b64decode(content.data)\n        pil_image = PILImage.open(io.BytesIO(image_data))\n        return Image.from_pil(pil_image)\n    else:\n        raise ValueError(f\"Unsupported content type: {content.type}\")\n\n\ndef parse_sampling_message(message: mcp_types.SamplingMessage, model_info: ModelInfo | None = None) -> LLMMessage:\n    \"\"\"Convert MCP sampling messages to AutoGen LLM messages.\n\n    Args:\n        message: MCP sampling message with role and content\n        model_info: Optional model information for content parsing\n\n    Returns:\n        Converted AutoGen LLM message (UserMessage or AssistantMessage)","sourceCodeStart":30,"sourceCodeEnd":66,"githubUrl":"https://github.com/microsoft/autogen/blob/027ecf0a379bcc1d09956d46d12d44a3ad9cee14/python/packages/autogen-ext/src/autogen_ext/tools/mcp/_host/_sampling.py#L30-L66","documentation":"When an MCP client requests server-side sampling with image content, the host converts it to AutoGen's Image type. If the configured model_info says the model family lacks vision support, RuntimeError('model {family} does not support vision.') is raised rather than sending an unusable request. model_info comes from the chat model client the sampling host was configured with.","triggerScenarios":"An MCP server sends a sampling request containing content with type='image' while the host's model_info has vision=False (or the family's capability entry omits vision). parse_content is called per content block during parse_sampling_message.","commonSituations":"Using a text-only model (e.g. a completion-style or small local model) with an MCP server that attaches screenshots/images; model_info defaults being conservative (vision not set) for a custom or newly added model family; a custom model_info dict built by hand without the vision key.","solutions":["Switch the sampling host's chat model to a vision-capable model (e.g. gpt-4o family).","If the model genuinely supports images, provide a correct model_info dict with vision=True (and the right family) when constructing the model client.","On the server side, stop sending image content to non-vision models, or make image blocks optional."],"exampleFix":"# before\nclient = OpenAIChatCompletionClient(model=\"gpt-3.5-turbo\")  # vision=False\n# MCP server sampling request with image -> RuntimeError\n\n# after\nclient = OpenAIChatCompletionClient(\n    model=\"gpt-4o\",\n    model_info={\n        \"vision\": True,\n        \"function_calling\": True,\n        \"json_output\": True,\n        \"family\": ModelFamily.GPT_4O,\n    },\n)","handlingStrategy":"validation","validationCode":"info = client.model_info or {}\nhas_image = any(b.get(\"type\") == \"image\" for b in message_content)\nif has_image and not info.get(\"vision\", False):\n    raise ValueError(\"reject sampling request: model lacks vision\")","typeGuard":"def model_supports_vision(model_info: dict | None) -> bool:\n    return bool(model_info and model_info.get(\"vision\", False))","tryCatchPattern":"try:\n    msg = parse_sampling_message(message, model_info)\nexcept RuntimeError as e:\n    if \"does not support vision\" in str(e):\n        return error_result(\"vision not supported by this model\")\n    raise","preventionTips":["Declare accurate model_info (vision flag) for every custom model client.","Advertise vision capability in the MCP sampling context so servers do not send images to text-only models.","Test sampling with an image message whenever you change model families."],"tags":["mcp","sampling","vision","multimodal","model-capabilities"],"backgroundTag":null,"analyzedSha":"027ecf0a379bcc1d09956d46d12d44a3ad9cee14","analyzedAt":"2026-08-15T03:38:00.719Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}