run-llama/llama_index · error · OutputParserException

Failed to parse pydantic object from guidance program. Proba

Error message

Failed to parse pydantic object from guidance program. Probably the LLM failed to produce data with right json schema

What it means

OutputParserException from parse_pydantic_from_guidance_program: after a guidance program executes, llama-index re-extracts the JSON by splitting on the '```json' fence and running parse_json_markdown, then validates it against your pydantic class. Any failure anywhere in that chain (no fence found, truncated JSON, wrong field names/types) is swallowed and re-raised as this single generic message with the original exception chained via `from e`.

Source

Thrown at llama-index-core/llama_index/core/prompts/guidance_utils.py:155

    This is a temporary solution for parsing a pydantic object out of an executed
    guidance program.

    NOTE: right now we assume the output is the last markdown formatted json block

    NOTE: a better way is to extract via Program.variables, but guidance does not
          support extracting nested objects right now.
          So we call back to manually parsing the final text after program execution
    """
    try:
        output = response.split("```json")[-1]
        output = "```json" + output
        if verbose:
            print("Raw output:")
            print(output)
        json_dict = parse_json_markdown(output)
        sub_questions = cls.model_validate(json_dict)
    except Exception as e:
        raise OutputParserException(
            "Failed to parse pydantic object from guidance program"
            ". Probably the LLM failed to produce data with right json schema"
        ) from e
    return sub_questions

View on GitHub (pinned to afd0fef371)

Solutions

Re-run with verbose=True in parse (or print the raw program output) to see exactly what the LLM produced — the chained exception (`__cause__`) usually names the real pydantic validation error.
Raise the LLM's max_tokens / num_output so the JSON block is never truncated.
Make the pydantic model lenient: give fields defaults, make them Optional with validators, so minor schema drift still validates.
Switch to a model/provider that follows the guidance template reliably, or move to native function-calling structured output where correctness is enforced by the API.

Example fix

// before
result = program(..., verbose=False)  # fails opaquely

// after
from llama_index.core.output_parsers import OutputParserException
try:
    result = program(..., verbose=True)  # prints raw output before parsing
except OutputParserException as e:
    print("cause:", e.__cause__)         # real pydantic/json error
    raise

Defensive patterns

Strategy: try-catch

Validate before calling

def looks_like_guidance_json(response: str) -> bool:
    """Cheap pre-check that the executed program emitted a fenced JSON block."""
    return "```json" in response and response.rstrip().endswith("```")

Try / catch

from llama_index.core.output_parsers import OutputParserException

for attempt in range(3):
    try:
        result = program(dry_run=False, verbose=True)
        break
    except OutputParserException as e:
        if attempt == 2:
            raise
        cause = e.__cause__ or e
        print(f"attempt {attempt} failed: {cause}")  # real pydantic/json error
        # optionally raise max_tokens or simplify the model, then retry

Prevention

Run with verbose=True while developing to see the raw LLM output before parsing.
Inspect e.__cause__ — the chained exception carries the actual pydantic validation error.
Give every output-model field a default so partially-formed JSON still validates.
Set generous max_tokens; truncation is the most common cause of unparseable fenced JSON.

When it happens

Trigger: GuidancePydanticProgram(...) where the executed program output lacks a '```json' block, the JSON is malformed/truncated by a token limit, the LLM emitted fields that fail cls.model_validate (missing required fields, wrong types), or the whole response is empty because the guidance template never executed (misconfigured llm).

Common situations: Low max_tokens cutting the JSON mid-object; using a weak local model that does not respect the guidance template; pydantic v1 vs v2 mismatch between the model and validation call; prompt/template edits that break the markdown fence the parser hard-codes on.

Understand the failure class

Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

AI-assisted analysis of run-llama/llama_index@afd0fef371 (2026-08-15). Data as JSON: /api/errors/47e8bc004103fce8. Report an issue: GitHub.