run-llama/llama_index · error · TypeError

StructuredLLM expected a {self.output_cls.__name__} instance

Error message

StructuredLLM expected a {self.output_cls.__name__} instance from astructured_predict, but got {type(output).__name__}: {output!r}. The underlying LLM failed to produce valid structured output.

What it means

Async counterpart of the StructuredLLM type check: astructured_predict must return a pydantic BaseModel instance; when the underlying LLM's structured output machinery yields anything else (dict, str, None), StructuredLLM.achat raises TypeError, flagging that the model failed to produce valid structured output for output_cls.

Source

Thrown at llama-index-core/llama_index/core/llms/structured_llm.py:126

    # ===== Async Endpoints =====
    @llm_chat_callback()
    async def achat(
        self,
        messages: Sequence[ChatMessage],
        **kwargs: Any,
    ) -> ChatResponse:
        # NOTE: we are wrapping existing messages in a ChatPromptTemplate to
        # make this work with our FunctionCallingProgram, even though
        # the messages don't technically have any variables (they are already formatted)

        chat_prompt = ChatPromptTemplate(message_templates=messages)

        output = await self.llm.astructured_predict(
            output_cls=self.output_cls, prompt=chat_prompt, llm_kwargs=kwargs
        )
        if not isinstance(output, BaseModel):
            raise TypeError(
                f"StructuredLLM expected a {self.output_cls.__name__} instance "
                f"from astructured_predict, but got {type(output).__name__}: "
                f"{output!r}. The underlying LLM failed to produce valid "
                f"structured output."
            )
        return ChatResponse(
            message=ChatMessage(
                role=MessageRole.ASSISTANT, content=output.model_dump_json()
            ),
            raw=output,
        )

    @llm_chat_callback()
    async def astream_chat(
        self,
        messages: Sequence[ChatMessage],
        **kwargs: Any,
    ) -> ChatResponseAsyncGen:

View on GitHub (pinned to afd0fef371)

Solutions

1. Define output_cls as a pydantic v2 BaseModel.
2. Switch to a model with reliable native structured output (function calling, JSON mode, or grammar-constrained decoding).
3. Simplify the schema and add descriptions/defaults to make compliance easier.
4. Wrap achat in try/except TypeError with one retry; transient parse failures are common.

Example fix

# before
resp = await structured_llm.achat(msgs)
# after
from pydantic import BaseModel
class Answer(BaseModel):
    text: str
structured_llm = llm.as_structured_llm(Answer)
try:
    resp = await structured_llm.achat(msgs)
except TypeError:
    resp = await structured_llm.achat(msgs)  # one retry

from pydantic import BaseModel

def is_valid_output_cls(output_cls) -> bool:
    return isinstance(output_cls, type) and issubclass(output_cls, BaseModel)

from pydantic import BaseModel

def is_pydantic_v2_model(cls) -> bool:
    return isinstance(cls, type) and issubclass(cls, BaseModel) and hasattr(cls, "model_validate")

for attempt in range(2):
    try:
        resp = await structured_llm.achat(messages)
        break
    except TypeError as e:
        if "StructuredLLM expected" not in str(e) or attempt == 1:
            raise

When it happens

Trigger: Awaiting structured_llm.achat(messages) (or acomplete, which delegates to achat) when the wrapped LLM returns a non-BaseModel from astructured_predict — no function calling, invalid JSON, or output_cls not being a pydantic v2 model.

Common situations: Async chat apps using as_structured_llm with local/open-weight models; mixing pydantic v1 schemas with a pydantic v2 llama-index-core; overly complex output schemas the model cannot satisfy.

Related errors

• StructuredLLM expected a {self.output_cls.__name__} instance
• astructured_predict expected a {output_cls.__name__} instanc
• astream_complete is not supported by default.
• key should not be None
• Aborting parsing document; {numTags} elements found

AI-assisted analysis of run-llama/llama_index@afd0fef371 (2026-08-15). Data as JSON: /api/errors/f282d160a80dcc2f. Report an issue: GitHub.