run-llama/llama_index · error · TypeError

StructuredLLM expected a {self.output_cls.__name__} instance

Error message

StructuredLLM expected a {self.output_cls.__name__} instance from structured_predict, but got {type(output).__name__}: {output!r}. The underlying LLM failed to produce valid structured output.

What it means

StructuredLLM (from llm.as_structured_llm(...)) wraps the underlying LLM's structured_predict and asserts the result is a pydantic BaseModel. If the wrapped LLM's structured-prediction machinery returns anything else (a plain dict, a string, None), it means the model output could not be coerced into the target schema, and StructuredLLM raises TypeError rather than emitting a chat response with an unusable payload.

Source

Thrown at llama-index-core/llama_index/core/llms/structured_llm.py:65

    @property
    def metadata(self) -> LLMMetadata:
        return self.llm.metadata

    @llm_chat_callback()
    def chat(self, messages: Sequence[ChatMessage], **kwargs: Any) -> ChatResponse:
        """Chat endpoint for LLM."""
        # NOTE: we are wrapping existing messages in a ChatPromptTemplate to
        # make this work with our FunctionCallingProgram, even though
        # the messages don't technically have any variables (they are already formatted)

        chat_prompt = ChatPromptTemplate(message_templates=messages)

        output = self.llm.structured_predict(
            output_cls=self.output_cls, prompt=chat_prompt, llm_kwargs=kwargs
        )
        if not isinstance(output, BaseModel):
            raise TypeError(
                f"StructuredLLM expected a {self.output_cls.__name__} instance "
                f"from structured_predict, but got {type(output).__name__}: "
                f"{output!r}. The underlying LLM failed to produce valid "
                f"structured output."
            )
        return ChatResponse(
            message=ChatMessage(
                role=MessageRole.ASSISTANT, content=output.model_dump_json()
            ),
            raw=output,
        )

    @llm_chat_callback()
    def stream_chat(
        self, messages: Sequence[ChatMessage], **kwargs: Any
    ) -> ChatResponseGen:
        chat_prompt = ChatPromptTemplate(message_templates=messages)

View on GitHub (pinned to afd0fef371)

Solutions

1. Make output_cls a pydantic v2 BaseModel (inherit from pydantic.BaseModel) so isinstance(output, BaseModel) can pass.
2. Use a model with native function calling / structured output (OpenAI, Anthropic) or enable the integration's JSON/grammar mode.
3. Simplify the schema (fewer/optional fields) and add field descriptions so the model can comply.
4. Catch the TypeError and retry the chat call — transient malformed outputs often succeed on retry.

Example fix

# before
from dataclasses import dataclass
@dataclass
class Answer: ...  # not a pydantic model -> isinstance fails
sllm.chat(msgs)
# after
from pydantic import BaseModel
class Answer(BaseModel):
    text: str
sllm.chat(msgs)

from pydantic import BaseModel

def is_valid_output_cls(output_cls) -> bool:
    return isinstance(output_cls, type) and issubclass(output_cls, BaseModel)

from pydantic import BaseModel

def is_pydantic_v2_model(cls) -> bool:
    return isinstance(cls, type) and issubclass(cls, BaseModel) and hasattr(cls, "model_validate")

try:
    resp = structured_llm.chat(messages)
except TypeError as e:
    if "StructuredLLM expected" in str(e):
        resp = structured_llm.chat(messages)  # single retry; transient parse failures are common
    else:
        raise

When it happens

Trigger: Calling structured_llm.chat(messages) (or complete(), which delegates to chat) where the underlying LLM cannot produce valid structured output for output_cls — weak models, missing function-calling support, or an output_cls that is not a pydantic v2 BaseModel so isinstance() fails.

Common situations: Using a small/local model (llama.cpp, Ollama without native structured output) with as_structured_llm; defining output_cls as a dataclass or pydantic v1 model while llama-index-core expects pydantic v2 BaseModel; prompt/schema so complex the model returns prose instead of JSON.

Related errors

• StructuredLLM expected a {self.output_cls.__name__} instance
• key should not be None
• Failed to validate query spec. Error: {e}. Got JSON dict: {j
• structured_predict expected a {output_cls.__name__} instance
• astructured_predict expected a {output_cls.__name__} instanc

AI-assisted analysis of run-llama/llama_index@afd0fef371 (2026-08-15). Data as JSON: /api/errors/90be8d6aaa152cbd. Report an issue: GitHub.