run-llama/llama_index · error · TypeError

astructured_predict expected a {output_cls.__name__} instanc

Error message

astructured_predict expected a {output_cls.__name__} instance but got {type(result).__name__}: {result!r}. The LLM failed to produce valid structured output.

What it means

Async counterpart of structured_predict: astructured_predict() awaits program.acall() and then requires a pydantic BaseModel instance. If the program yields a non-BaseModel (string, dict, None), the LLM failed to generate valid structured output and this TypeError is raised with the offending value.

Source

Thrown at llama-index-core/llama_index/core/llms/llm.py:432

        dispatcher.event(
            LLMStructuredPredictStartEvent(
                output_cls=output_cls, template=prompt, template_args=prompt_args
            )
        )

        program = get_program_for_llm(
            output_cls,
            prompt,
            self,
            pydantic_program_mode=self.pydantic_program_mode,
        )

        result = await program.acall(llm_kwargs=llm_kwargs, **prompt_args)
        assert not isinstance(result, list)

        if not isinstance(result, BaseModel):
            raise TypeError(
                f"astructured_predict expected a {output_cls.__name__} instance "
                f"but got {type(result).__name__}: {result!r}. "
                f"The LLM failed to produce valid structured output."
            )

        dispatcher.event(LLMStructuredPredictEndEvent(output=result))
        return result

    def _structured_stream_call(
        self,
        output_cls: Type[Model],
        prompt: PromptTemplate,
        llm_kwargs: Optional[Dict[str, Any]] = None,
        **prompt_args: Any,
    ) -> Generator[
        Union[Model, List[Model], "FlexibleModel", List["FlexibleModel"]], None, None
    ]:
        from llama_index.core.program.utils import get_program_for_llm

View on GitHub (pinned to afd0fef371)

Solutions

1. Switch to a function-calling LLM or a program mode that enforces the schema.
2. Verify output_cls subclasses pydantic.BaseModel.
3. Add a bounded retry around astructured_predict for transient malformed output.
4. Inspect the reported {result!r} to tune the prompt (e.g. demand raw JSON only, no markdown fences).

Example fix

# before
result = await llm.astructured_predict(Output, prompt_tmpl, query=q)  # raises TypeError on bad JSON

# after
for attempt in range(3):
    try:
        result = await llm.astructured_predict(Output, prompt_tmpl, query=q)
        break
    except TypeError:
        if attempt == 2:
            raise

from pydantic import BaseModel
assert issubclass(OutputCls, BaseModel), 'output_cls must be a pydantic BaseModel'

from pydantic import BaseModel

def is_pydantic_model(cls) -> bool:
    return isinstance(cls, type) and issubclass(cls, BaseModel)

for attempt in range(3):
    try:
        result = await llm.astructured_predict(Output, prompt, **kw)
        break
    except TypeError:
        if attempt == 2:
            raise

When it happens

Trigger: await llm.astructured_predict(OutputModel, prompt, ...) with a non-function-calling model whose output fails JSON parsing; output_cls that is not a pydantic class; a custom async program returning raw text; truncated JSON from exceeding the context window.

Common situations: Async services (FastAPI backends) doing structured extraction with local/open-source models; occasional malformed JSON under load; switching from OpenAI to a provider without native JSON mode.

Related errors

• structured_predict expected a {output_cls.__name__} instance
• StructuredLLM expected a {self.output_cls.__name__} instance
• Expected ActionReasoningStep, got {reasoning_step}
• StructuredLLM expected a {self.output_cls.__name__} instance
• astream_complete is not supported by default.

AI-assisted analysis of run-llama/llama_index@afd0fef371 (2026-08-15). Data as JSON: /api/errors/c0f5119503546973. Report an issue: GitHub.