{"record":{"id":"3a0e310e9481a18e","repo":"run-llama/llama_index","slug":"multimodal-synthesis-requires-a-chat-llm","errorCode":null,"errorMessage":"Multimodal synthesis requires a chat LLM.","messagePattern":"Multimodal synthesis requires a chat LLM\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"llama-index-core/llama_index/core/response_synthesizers/base.py","lineNumber":92,"sourceCode":"        output_cls: Optional[Type[BaseModel]] = None,\n        empty_response: Optional[str] = None,\n        multimodal: bool = False,\n    ) -> None:\n        \"\"\"Init params.\"\"\"\n        self._llm = llm or Settings.llm\n\n        if callback_manager:\n            self._llm.callback_manager = callback_manager\n\n        self._callback_manager = callback_manager or Settings.callback_manager\n        self._streaming = streaming\n        self._output_cls = output_cls\n        self._empty_response = empty_response or \"Empty Response\"\n        self._multimodal = multimodal\n        self._prompt_helper: PromptHelper\n        if multimodal:\n            if not is_chat_model(self._llm):\n                raise ValueError(\"Multimodal synthesis requires a chat LLM.\")\n        self._prompt_helper = (\n            prompt_helper\n            or Settings._prompt_helper\n            or PromptHelper.from_llm_metadata(\n                self._llm.metadata,\n            )\n        )\n        self._chat_prompt_helper = (\n            chat_prompt_helper\n            or Settings._chat_prompt_helper\n            or ChatPromptHelper.from_llm_metadata(\n                self._llm.metadata,\n            )\n        )\n\n    def _empty_response_generator(self) -> Generator[str, None, None]:\n        yield self._empty_response\n","sourceCodeStart":74,"sourceCodeEnd":110,"githubUrl":"https://github.com/run-llama/llama_index/blob/afd0fef371831f9bda13e5af7167cf4e981278ab/llama-index-core/llama_index/core/response_synthesizers/base.py#L74-L110","documentation":"BaseSynthesizer.__init__ with multimodal=True requires the configured LLM to be a chat model (is_chat_model check), because multimodal synthesis sends image content via chat messages, which completion-style LLMs cannot express. The check runs against self._llm, which may come from the constructor argument or Settings.llm.","triggerScenarios":"Building a query engine or ResponseSynthesizer with multimodal=True while Settings.llm (or the passed llm) is a completion model such as an OpenAI 'text-davinci-*'-style or HuggingFaceHub completion LLM; setting multimodal=True in a RetrieverQueryEngine factory; a default env where a non-chat LLM was installed globally.","commonSituations":"Multimodal RAG demos that forget to pass llm=OpenAI(model='gpt-4o'); environments where Settings.llm was set to a legacy completion model earlier; integration tests with a stub completion LLM.","solutions":["Pass a chat LLM explicitly: ResponseSynthesizer(multimodal=True, llm=OpenAI(model='gpt-4o-mini')).","Set Settings.llm to a chat model (any LLM class whose is_chat_model is True) before constructing the engine.","If you do not need image input, drop multimodal=True.","Verify with llama_index.core.utils.is_chat_model(Settings.llm) at startup."],"exampleFix":"# before\nsynth = ResponseSynthesizer(multimodal=True)  # Settings.llm is a completion model\n\n# after\nfrom llama_index.llms.openai import OpenAI\nsynth = ResponseSynthesizer(multimodal=True, llm=OpenAI(model=\"gpt-4o\"))","handlingStrategy":"validation","validationCode":"from llama_index.core.utils import is_chat_model\nassert is_chat_model(llm), f\"multimodal synthesis requires a chat LLM, got {type(llm)}\"","typeGuard":"from llama_index.core.llms import ChatLLM\nfrom llama_index.core.utils import is_chat_model\n\ndef is_chat_llm(llm) -> bool:\n    return is_chat_model(llm)","tryCatchPattern":null,"preventionTips":["Always pass an explicit chat LLM when multimodal=True.","Check is_chat_model(Settings.llm) during app startup."],"tags":["multimodal","llm","configuration","initialization"],"backgroundTag":null,"analyzedSha":"afd0fef371831f9bda13e5af7167cf4e981278ab","analyzedAt":"2026-08-15T05:42:58.429Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}