{"record":{"id":"f7ba107de36d73b5","repo":"run-llama/llama_index","slug":"stream-complete-is-not-supported-by-default","errorCode":null,"errorMessage":"stream_complete is not supported by default.","messagePattern":"stream_complete is not supported by default\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"llama-index-core/llama_index/core/llms/structured_llm.py","lineNumber":107,"sourceCode":"                message=ChatMessage(\n                    role=MessageRole.ASSISTANT, content=partial_output.json()\n                ),\n                raw=partial_output,\n            )\n\n    @llm_completion_callback()\n    def complete(\n        self, prompt: str, formatted: bool = False, **kwargs: Any\n    ) -> CompletionResponse:\n        complete_fn = chat_to_completion_decorator(self.chat)\n        return complete_fn(prompt, **kwargs)\n\n    @llm_completion_callback()\n    def stream_complete(\n        self, prompt: str, formatted: bool = False, **kwargs: Any\n    ) -> CompletionResponseGen:\n        \"\"\"Stream completion endpoint for LLM.\"\"\"\n        raise NotImplementedError(\"stream_complete is not supported by default.\")\n\n    # ===== Async Endpoints =====\n    @llm_chat_callback()\n    async def achat(\n        self,\n        messages: Sequence[ChatMessage],\n        **kwargs: Any,\n    ) -> ChatResponse:\n        # NOTE: we are wrapping existing messages in a ChatPromptTemplate to\n        # make this work with our FunctionCallingProgram, even though\n        # the messages don't technically have any variables (they are already formatted)\n\n        chat_prompt = ChatPromptTemplate(message_templates=messages)\n\n        output = await self.llm.astructured_predict(\n            output_cls=self.output_cls, prompt=chat_prompt, llm_kwargs=kwargs\n        )\n        if not isinstance(output, BaseModel):","sourceCodeStart":89,"sourceCodeEnd":125,"githubUrl":"https://github.com/run-llama/llama_index/blob/afd0fef371831f9bda13e5af7167cf4e981278ab/llama-index-core/llama_index/core/llms/structured_llm.py#L89-L125","documentation":"StructuredLLM implements chat/complete by round-tripping through structured_predict, which is an inherently non-incremental operation: the full JSON output must be validated before any text is emitted. Therefore stream_complete is intentionally left unimplemented and raises NotImplementedError by default.","triggerScenarios":"Calling structured_llm.stream_complete(prompt) directly, or routing a streaming pipeline (e.g. a query engine with streaming=True) through an LLM obtained via llm.as_structured_llm().","commonSituations":"Enabling streaming on an index/query engine whose Settings.llm was replaced by a structured LLM; building a chat UI that always consumes CompletionResponseGen and hitting the structured wrapper.","solutions":["Use the non-streaming structured_llm.complete(prompt) and emit the single validated result.","If you only need streaming chat, call the original unwrapped llm.stream_complete / stream_chat.","Wrap complete() in a generator that yields one CompletionResponse if your interface demands a stream.","Use native structured-output modes of the provider SDK directly when streaming JSON is a hard requirement."],"exampleFix":"# before\nresp_gen = structured_llm.stream_complete(prompt)  # raises\n# after\nresp = structured_llm.complete(prompt)\ndef one_shot_stream():\n    yield resp\nresp_gen = one_shot_stream()","handlingStrategy":"type-guard","validationCode":"from llama_index.core.llms.structured_llm import StructuredLLM\n\ndef supports_stream_complete(llm) -> bool:\n    from llama_index.core.llms.structured_llm import StructuredLLM as _S\n    return not isinstance(llm, _S)","typeGuard":"from llama_index.core.llms.structured_llm import StructuredLLM\n\ndef is_structured_llm(llm) -> bool:\n    return isinstance(llm, StructuredLLM)","tryCatchPattern":"try:\n    stream = structured_llm.stream_complete(prompt)\nexcept NotImplementedError:\n    resp = structured_llm.complete(prompt)\n    stream = iter([resp])  # adapt to stream-shaped consumer","preventionTips":["Never assume streaming on LLMs wrapped by as_structured_llm.","Keep a reference to the original LLM for streaming code paths.","Adapt non-streaming complete() into a single-item generator at your boundary."],"tags":["llama-index","structured-output","streaming","not-implemented"],"backgroundTag":null,"analyzedSha":"afd0fef371831f9bda13e5af7167cf4e981278ab","analyzedAt":"2026-08-15T05:42:58.429Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}