run-llama/llama_index · error · TypeError

structured_predict expected a {output_cls.__name__} instance

Error message

structured_predict expected a {output_cls.__name__} instance but got {type(result).__name__}: {result!r}. The LLM failed to produce valid structured output.

What it means

TypeError from the sync structured_predict(): after running the Pydantic program over the LLM output, llama-index asserts the result is a single pydantic BaseModel. If the program returned something else (a string, dict, list of models, or None), the LLM failed to produce valid structured output and the raw non-conforming result is surfaced in the message.

Source

Thrown at llama-index-core/llama_index/core/llms/llm.py:364

        from llama_index.core.program.utils import get_program_for_llm

        dispatcher.event(
            LLMStructuredPredictStartEvent(
                output_cls=output_cls, template=prompt, template_args=prompt_args
            )
        )
        program = get_program_for_llm(
            output_cls,
            prompt,
            self,
            pydantic_program_mode=self.pydantic_program_mode,
        )

        result = program(llm_kwargs=llm_kwargs, **prompt_args)
        assert not isinstance(result, list)

        if not isinstance(result, BaseModel):
            raise TypeError(
                f"structured_predict expected a {output_cls.__name__} instance "
                f"but got {type(result).__name__}: {result!r}. "
                f"The LLM failed to produce valid structured output."
            )

        dispatcher.event(LLMStructuredPredictEndEvent(output=result))
        return result

    @dispatcher.span
    async def astructured_predict(
        self,
        output_cls: Type[Model],
        prompt: PromptTemplate,
        llm_kwargs: Optional[Dict[str, Any]] = None,
        **prompt_args: Any,
    ) -> Model:
        r"""
        Async Structured predict.

View on GitHub (pinned to afd0fef371)

Solutions

1. Use an LLM with native structured output/function calling (e.g. OpenAI) or pass pydantic_program_mode=PydanticProgramMode.OPENAI / a custom program that validates.
2. Ensure output_cls is a pydantic BaseModel (not dataclass/TypedDict).
3. Retry — LLM structured output is nondeterministic; a retry loop with a stricter prompt often fixes one-off malformed JSON.
4. Log the {result!r} payload from the message to see exactly what the model returned, then adjust the prompt or parser.

Example fix

# before
class Output(BaseModel): ...
result = llm.structured_predict(Output, prompt_tmpl, query=q)  # local LLM emits prose -> TypeError

# after
from llama_index.core.program.pydantic_program_utils import PydanticProgramMode
result = llm.structured_predict(
    Output, prompt_tmpl, query=q,
    pydantic_program_mode=PydanticProgramMode.OPENAI,  # or a validating custom program
)

from pydantic import BaseModel
assert issubclass(OutputCls, BaseModel), 'output_cls must be a pydantic BaseModel'

from pydantic import BaseModel

def is_pydantic_model(cls) -> bool:
    return isinstance(cls, type) and issubclass(cls, BaseModel)

for attempt in range(3):
    try:
        result = llm.structured_predict(Output, prompt, **kw)
        break
    except TypeError:
        if attempt == 2:
            raise
        # tighten the prompt or bump temperature=0 retry

When it happens

Trigger: llm.structured_predict(OutputModel, PromptTemplate, ...) where the LLM output cannot be parsed into OutputModel — malformed JSON, an LLM without native function calling falling back to a text-parsing program, output_cls accidentally being a non-pydantic class so the program returns raw text, or a custom pydantic_program whose __call__ returns the wrong type.

Common situations: Using non-function-calling LLMs (local/open models) with the default program mode and sloppy JSON output; output_cls passed as a dataclass or TypedDict instead of a BaseModel; low temperature=0 prompts where the model emits prose around JSON; context-length overflow truncating the JSON.

Related errors

• Expected ActionReasoningStep, got {reasoning_step}
• astructured_predict expected a {output_cls.__name__} instanc
• StructuredLLM expected a {self.output_cls.__name__} instance
• StructuredLLM expected a {self.output_cls.__name__} instance
• Malformed partial JSON encountered.

AI-assisted analysis of run-llama/llama_index@afd0fef371 (2026-08-15). Data as JSON: /api/errors/2673913b153ad135. Report an issue: GitHub.