{"record":{"id":"79352d364fbdb03e","repo":"zylon-ai/private-gpt","slug":"llm-does-not-support-structured-chat-79352d","errorCode":null,"errorMessage":"LLM does not support structured chat.","messagePattern":"LLM does not support structured chat\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"private_gpt/components/multimodality/image_handler.py","lineNumber":365,"sourceCode":"            async with retry_context(\n                tries=self._num_max_retries,\n                jitter=self._retry_jitter,\n                logger=logger,\n            ) as retry:\n                seed = kwargs.pop(\"seed\", None) or 0\n                semaphore_manager: SemaphoreManager | None = kwargs.pop(\n                    \"semaphore_manager\", None\n                )\n                count = 0\n                max_iterations = kwargs.pop(\"max_iterations\", 3)\n\n                async def _call() -> Any:\n                    nonlocal count\n                    count += 1\n\n                    structured_chat = getattr(self._llm, \"astructured_chat\", None)\n                    if not callable(structured_chat):\n                        raise NotImplementedError(\n                            \"LLM does not support structured chat.\"\n                        )\n\n                    new_kwargs = kwargs.copy()\n                    new_kwargs[\"seed\"] = str(seed) + str(count)\n\n                    try:\n                        current_messages = messages\n                        if count > 1:\n                            current_messages = self._reduce_images_in_messages(\n                                messages, count - 1\n                            )\n                            logger.info(\n                                f\"Retry {count}: Reduced image quality (iteration {count - 1}/{max_iterations})\"\n                            )\n\n                        return await structured_chat(\n                            response_model, current_messages, **new_kwargs","sourceCodeStart":347,"sourceCodeEnd":383,"githubUrl":"https://github.com/zylon-ai/private-gpt/blob/4a030776a31a901ad80b1bf4d7faa2c1a367efbb/private_gpt/components/multimodality/image_handler.py#L347-L383","documentation":"Same capability check as the audio handler, but in the image handler's retried structured-chat loop: before each attempt (including retries after image-reduction passes) it verifies `callable(getattr(self._llm, 'astructured_chat', None))` and raises `NotImplementedError` if the LLM wrapper cannot produce Pydantic-validated structured output. It fires before any request is sent.","triggerScenarios":"Running image extraction/description pipelines with an LLM class lacking `astructured_chat`; first attempt and any retry (the `count > 1` image-reduction path) both re-enter `_call` and re-check; using a custom or stub LLM injected into the image handler.","commonSituations":"Configuring a chat-only or completion-only LLM backend for multimodal image work; test doubles that don't mirror the real LLM surface; llama-index version drift renaming structured-output methods; local models behind a minimal wrapper.","solutions":["Point the multimodal LLM setting at a provider that implements `astructured_chat`","Add/alias `astructured_chat` on the custom LLM wrapper (delegate to structured-output support or parse into the response model)","For tests, provide a fake LLM with an async `astructured_chat` method"],"exampleFix":"// before\nllm = CustomChatLLM()  # no astructured_chat\nhandler = ImageHandler(llm, ...)\nawait handler.extract(...)  # NotImplementedError\n\n// after\nclass CustomChatLLM:\n    async def astructured_chat(self, response_model, messages, **kwargs):\n        resp = await self.achat(messages)\n        return response_model.model_validate_json(resp.content)","handlingStrategy":"type-guard","validationCode":"if not callable(getattr(image_llm, \"astructured_chat\", None)):\n    raise ValueError(\"image LLM must implement astructured_chat\")","typeGuard":"def supports_structured_images(llm: Any) -> bool:\n    return callable(getattr(llm, \"astructured_chat\", None))","tryCatchPattern":"try:\n    await image_handler.extract(...)\nexcept NotImplementedError:\n    # permanent capability gap; fail fast, do not retry\n    raise","preventionTips":["Validate the multimodal LLM capability at wiring time (DI container), not at request time","Pin llama-index versions so structured-output method names stay stable","Document which providers support structured chat and enforce via a startup check"],"tags":["llm","structured-output","capability-check","multimodality","vision"],"backgroundTag":null,"analyzedSha":"4a030776a31a901ad80b1bf4d7faa2c1a367efbb","analyzedAt":"2026-08-15T03:51:26.951Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}