run-llama/llama_index · error · NotImplementedError

stream_complete is not supported by default.

Error message

stream_complete is not supported by default.

What it means

StructuredLLM implements chat/complete by round-tripping through structured_predict, which is an inherently non-incremental operation: the full JSON output must be validated before any text is emitted. Therefore stream_complete is intentionally left unimplemented and raises NotImplementedError by default.

Source

Thrown at llama-index-core/llama_index/core/llms/structured_llm.py:107

                message=ChatMessage(
                    role=MessageRole.ASSISTANT, content=partial_output.json()
                ),
                raw=partial_output,
            )

    @llm_completion_callback()
    def complete(
        self, prompt: str, formatted: bool = False, **kwargs: Any
    ) -> CompletionResponse:
        complete_fn = chat_to_completion_decorator(self.chat)
        return complete_fn(prompt, **kwargs)

    @llm_completion_callback()
    def stream_complete(
        self, prompt: str, formatted: bool = False, **kwargs: Any
    ) -> CompletionResponseGen:
        """Stream completion endpoint for LLM."""
        raise NotImplementedError("stream_complete is not supported by default.")

    # ===== Async Endpoints =====
    @llm_chat_callback()
    async def achat(
        self,
        messages: Sequence[ChatMessage],
        **kwargs: Any,
    ) -> ChatResponse:
        # NOTE: we are wrapping existing messages in a ChatPromptTemplate to
        # make this work with our FunctionCallingProgram, even though
        # the messages don't technically have any variables (they are already formatted)

        chat_prompt = ChatPromptTemplate(message_templates=messages)

        output = await self.llm.astructured_predict(
            output_cls=self.output_cls, prompt=chat_prompt, llm_kwargs=kwargs
        )
        if not isinstance(output, BaseModel):

View on GitHub (pinned to afd0fef371)

Solutions

1. Use the non-streaming structured_llm.complete(prompt) and emit the single validated result.
2. If you only need streaming chat, call the original unwrapped llm.stream_complete / stream_chat.
3. Wrap complete() in a generator that yields one CompletionResponse if your interface demands a stream.
4. Use native structured-output modes of the provider SDK directly when streaming JSON is a hard requirement.

Example fix

# before
resp_gen = structured_llm.stream_complete(prompt)  # raises
# after
resp = structured_llm.complete(prompt)
def one_shot_stream():
    yield resp
resp_gen = one_shot_stream()

from llama_index.core.llms.structured_llm import StructuredLLM

def supports_stream_complete(llm) -> bool:
    from llama_index.core.llms.structured_llm import StructuredLLM as _S
    return not isinstance(llm, _S)

from llama_index.core.llms.structured_llm import StructuredLLM

def is_structured_llm(llm) -> bool:
    return isinstance(llm, StructuredLLM)

try:
    stream = structured_llm.stream_complete(prompt)
except NotImplementedError:
    resp = structured_llm.complete(prompt)
    stream = iter([resp])  # adapt to stream-shaped consumer

When it happens

Trigger: Calling structured_llm.stream_complete(prompt) directly, or routing a streaming pipeline (e.g. a query engine with streaming=True) through an LLM obtained via llm.as_structured_llm().

Common situations: Enabling streaming on an index/query engine whose Settings.llm was replaced by a structured LLM; building a chat UI that always consumes CompletionResponseGen and hitting the structured wrapper.

Related errors

• astream_complete is not supported by default.
• Output parser is not supported for streaming.
• Malformed partial JSON encountered.
• Streaming is not enabled. Please use chat() instead.
• Streaming is not enabled. Please use achat() instead.

AI-assisted analysis of run-llama/llama_index@afd0fef371 (2026-08-15). Data as JSON: /api/errors/f7ba107de36d73b5. Report an issue: GitHub.