{"record":{"id":"088ca07bee3cb91d","repo":"microsoft/semantic-kernel","slug":"the-model-does-not-support-multi-modality","errorCode":null,"errorMessage":"The model does not support multi-modality","messagePattern":"The model does not support multi-modality","errorType":"exception","errorClass":"ServiceInvalidExecutionSettingsError","httpStatus":null,"severity":"error","filePath":"python/semantic_kernel/connectors/ai/onnx/services/onnx_gen_ai_chat_completion.py","lineNumber":191,"sourceCode":"        return self.tokenizer.apply_chat_template(\n            json.dumps(self._chat_messages_to_dicts(chat_history)),\n            add_generation_prompt=True,\n        )\n\n    def _chat_messages_to_dicts(self, chat_history: \"ChatHistory\") -> list[dict[str, Any]]:\n        return [\n            message.to_dict(role_key=\"role\", content_key=\"content\")\n            for message in chat_history.messages\n            if isinstance(message, ChatMessageContent)\n        ]\n\n    def _get_images_from_history(self, chat_history: \"ChatHistory\") -> list[ImageContent] | None:\n        images = []\n        for message in chat_history.messages:\n            for image in message.items:\n                if isinstance(image, ImageContent):\n                    if not self.enable_multi_modality:\n                        raise ServiceInvalidExecutionSettingsError(\"The model does not support multi-modality\")\n                    if image.uri:\n                        images.append(image)\n                    else:\n                        raise ServiceInvalidExecutionSettingsError(\n                            \"Image Content URI needs to be set, because onnx can only work with file paths\"\n                        )\n        return images if len(images) else None\n\n    def _get_audios_from_history(self, chat_history: \"ChatHistory\") -> list[AudioContent] | None:\n        audios = []\n        for message in chat_history.messages:\n            for audio in message.items:\n                if isinstance(audio, AudioContent):\n                    if not self.enable_multi_modality:\n                        raise ServiceInvalidExecutionSettingsError(\"The model does not support multi-modality\")\n                    if audio.uri:\n                        audios.append(audio)\n                    else:","sourceCodeStart":173,"sourceCodeEnd":209,"githubUrl":"https://github.com/microsoft/semantic-kernel/blob/c028a0c7dc4f0814cdcbaba9d998f187a41197bf/python/semantic_kernel/connectors/ai/onnx/services/onnx_gen_ai_chat_completion.py#L173-L209","documentation":"Raised inside _get_images_from_history when chat_history contains an ImageContent item but the loaded model is not multi-modal (self.enable_multi_modality is False). The ONNX model was loaded without a 'vision' key in its config, so it cannot process images. Raised as ServiceInvalidExecutionSettingsError during the chat completion call.","triggerScenarios":"Calling get_chat_message_contents on a text-only ONNX model with chat_history that includes one or more ImageContent items. The check fires per-message during _get_images_from_history iteration.","commonSituations":"Switching from a vision model to a text-only model but keeping image inputs in the pipeline; adding image content to a chat that was designed for text-only; using the wrong model folder that happens to be text-only.","solutions":["Use a multi-modal (vision) ONNX model if you need image inputs","Remove ImageContent items from the chat history before sending to a text-only model","Switch to OnnxGenAIChatCompletion with a vision model and a template"],"exampleFix":"// before (text-only model + image in history)\nhistory.add_message(ChatMessageContent(role=AuthorRole.USER, items=[ImageContent(uri='...')]))\nawait chat.get_chat_message_contents(chat_history=history, settings=s)\n// after (remove image or use vision model)\nhistory.add_message(ChatMessageContent(role=AuthorRole.USER, items=[TextContent(text='describe the scene')]))","handlingStrategy":"validation","validationCode":"from semantic_kernel.contents import ImageContent\n\ndef check_images_supported(chat_service, chat_history):\n    if not chat_service.enable_multi_modality:\n        for msg in chat_history.messages:\n            if any(isinstance(item, ImageContent) for item in msg.items):\n                raise ValueError('Chat history contains images but model is not multi-modal')\n\ncheck_images_supported(chat, history)","typeGuard":"from semantic_kernel.contents import ImageContent\n\ndef history_has_images(chat_history) -> bool:\n    return any(\n        isinstance(item, ImageContent)\n        for msg in chat_history.messages\n        for item in msg.items\n    )","tryCatchPattern":"from semantic_kernel.exceptions.service_exceptions import ServiceInvalidExecutionSettingsError\n\ntry:\n    result = await chat.get_chat_message_contents(chat_history=history, settings=settings)\nexcept ServiceInvalidExecutionSettingsError as e:\n    if 'does not support multi-modality' in str(e):\n        logger.error('Remove image content or switch to a vision model')\n        raise","preventionTips":["Check chat.enable_multi_modality before adding images to chat history","Gate multimodal inputs behind a feature flag tied to the model type","Log the model type at construction so you know its capabilities"],"tags":["onnx","multimodal","image-content","execution-settings"],"backgroundTag":null,"analyzedSha":"c028a0c7dc4f0814cdcbaba9d998f187a41197bf","analyzedAt":"2026-08-13T13:48:05.040Z","schemaVersion":2},"datasetVersion":"2026-08-13T14:17:21.547Z"}