run-llama/llama_index · error · ValueError

No valid JSON found in output: {output}

Error message

No valid JSON found in output: {output}

What it means

SubQuestionOutputParser.parse() runs parse_json_markdown over the LLM output and expects a JSON array/object of sub-questions. If no JSON can be extracted (falsy result), it raises ValueError('No valid JSON found in output: {output}') before validating SubQuestion items (an 'items' wrapper key is unwrapped first for models like Zephyr).

Source

Thrown at llama-index-core/llama_index/core/question_gen/output_parser.py:13

from typing import Any

from llama_index.core.output_parsers.base import StructuredOutput
from llama_index.core.output_parsers.utils import parse_json_markdown
from llama_index.core.question_gen.types import SubQuestion
from llama_index.core.types import BaseOutputParser


class SubQuestionOutputParser(BaseOutputParser):
    def parse(self, output: str) -> Any:
        json_dict = parse_json_markdown(output)
        if not json_dict:
            raise ValueError(f"No valid JSON found in output: {output}")

        # example code includes an 'items' key, which breaks
        # the parsing from open-source LLMs such as Zephyr.
        # This gets the actual subquestions and recommended tools directly
        if "items" in json_dict:
            json_dict = json_dict["items"]

        sub_questions = [SubQuestion.model_validate(item) for item in json_dict]
        return StructuredOutput(raw_output=output, parsed_output=sub_questions)

    def format(self, prompt_template: str) -> str:
        return prompt_template

View on GitHub (pinned to afd0fef371)

Solutions

1. Use a stronger instruction-following LLM, or an OpenAI function-calling model via OpenAIQuestionGenerator
2. Raise max_tokens / output length so the JSON array is not truncated
3. If using a custom prompt_template_str, preserve the JSON structure and {output_cls} placeholder from the default
4. Catch ValueError in the query pipeline and retry the sub-question generation once

Example fix

// before
gen = LLMQuestionGenerator.from_defaults(llm=weak_local_llm)
sqe = SubQuestionQueryEngine(query_engine_tools=tools, question_gen=gen)
resp = sqe.query(q)  # raises on non-JSON output

// after
gen = LLMQuestionGenerator.from_defaults(
    llm=OpenAI(model="gpt-4o-mini")  # follows JSON format reliably
)
sqe = SubQuestionQueryEngine(query_engine_tools=tools, question_gen=gen)
resp = sqe.query(q)

# cheap pre-flight: ask the same model to emit JSON for a trivial prompt and try parsing it
from llama_index.core.output_parsers.utils import parse_json_markdown

probe = llm.complete("Return exactly this JSON: [{\"q\": \"hi\"}]")
if not parse_json_markdown(probe.text):
    raise RuntimeError("LLM cannot produce parseable JSON; sub-question parsing will fail")

try:
    resp = sub_question_engine.query(q)
except ValueError as e:
    if "No valid JSON found in output" in str(e):
        resp = sub_question_engine.query(q)  # retry once; JSON adherence is nondeterministic
    else:
        raise

When it happens

Trigger: LLMQuestionGenerator with a model whose output deviates from the DEFAULT_SUB_QUESTION_PROMPT format — pure prose, truncated JSON, or markdown that parse_json_markdown cannot recover JSON from.

Common situations: Small/local models (Zephyr, Llama variants) ignoring the JSON schema, aggressive max_tokens truncating the JSON, or a custom prompt that no longer asks for JSON.

Related errors

• Invalid JSON Path: {expression}
• The response is invalid
• The response is invalid
• Malformed partial JSON encountered.
• Failed to parse pydantic object from guidance program. Proba

AI-assisted analysis of run-llama/llama_index@afd0fef371 (2026-08-15). Data as JSON: /api/errors/594e8b6a5c26ca5a. Report an issue: GitHub.