run-llama/llama_index · error · NotImplementedError

astream_complete is not supported by default.

Error message

astream_complete is not supported by default.

What it means

StructuredLLM deliberately does not implement astream_complete: structured prediction validates the complete JSON payload before returning, so an async token stream cannot be produced. Calling it raises NotImplementedError.

Source

Thrown at llama-index-core/llama_index/core/llms/structured_llm.py:175

                    ),
                    raw=partial_output,
                )

        return gen()

    @llm_completion_callback()
    async def acomplete(
        self, prompt: str, formatted: bool = False, **kwargs: Any
    ) -> CompletionResponse:
        complete_fn = achat_to_completion_decorator(self.achat)
        return await complete_fn(prompt, **kwargs)

    @llm_completion_callback()
    async def astream_complete(
        self, prompt: str, formatted: bool = False, **kwargs: Any
    ) -> CompletionResponseGen:
        """Async stream completion endpoint for LLM."""
        raise NotImplementedError("astream_complete is not supported by default.")

View on GitHub (pinned to afd0fef371)

Solutions

1. Use await structured_llm.acomplete(prompt) for the single validated response.
2. Stream from the original unwrapped LLM when streaming matters more than schema validation.
3. Wrap acomplete's result in an async generator yielding one response if a stream-shaped API is required.

Example fix

# before
resp = await structured_llm.astream_complete(prompt)  # raises
# after
resp = await structured_llm.acomplete(prompt)
async def one_shot():
    yield resp
stream = one_shot()

from llama_index.core.llms.structured_llm import StructuredLLM

def supports_astream_complete(llm) -> bool:
    return not isinstance(llm, StructuredLLM)

from llama_index.core.llms.structured_llm import StructuredLLM

def is_structured_llm(llm) -> bool:
    return isinstance(llm, StructuredLLM)

try:
    stream = await structured_llm.astream_complete(prompt)
except NotImplementedError:
    resp = await structured_llm.acomplete(prompt)
    async def _one():
        yield resp
    stream = _one()

When it happens

Trigger: Awaiting/iterating structured_llm.astream_complete(prompt), or running an async streaming query engine whose LLM is a StructuredLLM wrapper.

Common situations: Async chat frontends that always consume async streams; setting Settings.llm = llm.as_structured_llm(...) globally and forgetting that some code paths stream.

Related errors

• stream_complete is not supported by default.
• Streaming is not enabled. Please use achat() instead.
• Output parser is not supported for streaming.
• StructuredLLM expected a {self.output_cls.__name__} instance
• Malformed partial JSON encountered.

AI-assisted analysis of run-llama/llama_index@afd0fef371 (2026-08-15). Data as JSON: /api/errors/af0b4d23db481841. Report an issue: GitHub.