{"record":{"id":"3a117c1c5fe877ed","repo":"google-gemini/gemini-cli","slug":"expected-json-object-from-llm-but-got-type-data","errorCode":null,"errorMessage":"Expected JSON object from LLM, but got {type(data).__name__}. Raw output:\\n{raw_text}","messagePattern":"Expected JSON object from LLM, but got (.+?)\\. Raw output:\\\\n(.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"tools/caretaker-agent/evals/triage/helpers/generate_golden_spec.py","lineNumber":44,"sourceCode":"from google.antigravity import Agent, LocalAgentConfig\nfrom google.antigravity.hooks.policy import deny\n\nPROMPT_FILE = Path(__file__).parent / \"generate_golden_spec.md\"\n\n\ndef _parse_llm_json(raw_text: str) -> dict:\n    \"\"\"Strips markdown fences and parses LLM JSON with fallback unescaping.\"\"\"\n    clean = raw_text.strip()\n    if clean.startswith(\"```\"):\n        clean = clean.split(\"\\n\", 1)[-1].rsplit(\"\\n\", 1)[0].strip()\n    try:\n        data = json.loads(clean, strict=False)\n    except Exception:\n        cleaned = re.sub(r'\\\\(?![/\"bfnrtu]|u[0-9a-fA-F]{4})', r'\\\\\\\\', re.sub(r\"(?<!\\\\)\\\\'\", \"'\", clean))\n        data = json.loads(cleaned, strict=False)\n\n    if not isinstance(data, dict):\n        raise ValueError(f\"Expected JSON object from LLM, but got {type(data).__name__}. Raw output:\\n{raw_text}\")\n\n    return data\n\n\ndef _load_system_instruction() -> str:\n    \"\"\"Loads prompt instructions from generate_golden_spec.md.\"\"\"\n    if not PROMPT_FILE.exists():\n        raise FileNotFoundError(f\"Required prompt file missing at: {PROMPT_FILE}\")\n    with open(PROMPT_FILE, \"r\", encoding=\"utf-8\") as f:\n        return f.read()\n\n\ndef generate_golden_spec(owner: str, repo: str, issue_number: int, issue_data: dict, pr_data: dict) -> dict:\n    \"\"\"\n    Invokes the Antigravity SDK (google.antigravity) Agent using generate_golden_spec.md\n    instructions to synthesize a clean, high-precision Workable Spec JSON and its rationale.\n    Returns a dict with keys: 'workable_spec' and 'golden_spec_rationale'.\n    \"\"\"","sourceCodeStart":26,"sourceCodeEnd":62,"githubUrl":"https://github.com/google-gemini/gemini-cli/blob/5024443c7217464a66e98f80d73172a26440bd8f/tools/caretaker-agent/evals/triage/helpers/generate_golden_spec.py#L26-L62","documentation":"This ValueError is raised in _parse_llm_json() when the Antigravity spec-generator LLM returns text that parses as valid JSON but is not a top-level JSON object (dict). After stripping markdown fences and applying a fallback unescape pass, if json.loads yields a list, string, number, bool, or None instead of a dict, the function rejects it. It guards the contract that the golden-spec agent must emit a structured object.","triggerScenarios":"The LLM emits a JSON array of items, a bare quoted string, or a number instead of an object. The agent wraps its answer in extra prose so the fence-stripping logic extracts the wrong segment. A model or SDK version change causes the response to be a top-level scalar.","commonSituations":"A prompt update removed the instruction to return a JSON object. The model returns a list because the prompt example used an array shape. The markdown fence stripper in _parse_llm_json mis-handles nested code blocks or a leading language tag, leaving non-JSON content that happens to parse as a non-dict type. extract_final_output concatenates chunks in an order that produces a fragment.","solutions":["Log or print raw_text before parsing to see exactly what the LLM returned and where it diverges from an object.","Update generate_golden_spec.md to explicitly require a top-level JSON object and include a conforming example.","If the model persistently returns a list, wrap the expectation: data = data[0] if isinstance(data, list) and data and isinstance(data[0], dict) else data, then re-validate.","Verify extract_final_output(resolved_chunks) returns the complete final agent message and not a truncated or multi-part stream."],"exampleFix":"# before: model returns [\"workable_spec\", {...}]\n# fix prompt in generate_golden_spec.md to show:\n# Respond with a single JSON object, e.g.:\n# {\"workable_spec\": {...}, \"golden_spec_rationale\": \"...\"}","handlingStrategy":"type-guard","validationCode":"import json, re\n\ndef safe_parse_llm_json(raw_text: str) -> dict:\n    clean = raw_text.strip()\n    if clean.startswith('```'):\n        clean = clean.split('\\n', 1)[-1].rsplit('\\n', 1)[0].strip()\n    try:\n        data = json.loads(clean, strict=False)\n    except Exception:\n        cleaned = re.sub(r'\\\\(?![/\"bfnrtu]|u[0-9a-fA-F]{4})', r'\\\\\\\\', re.sub(r\"(?<!\\\\\\\\)\\\\'\", \"'\", clean))\n        data = json.loads(cleaned, strict=False)\n    if isinstance(data, list) and data and isinstance(data[0], dict):\n        return data[0]\n    if not isinstance(data, dict):\n        raise ValueError(f'Expected JSON object, got {type(data).__name__}')\n    return data","typeGuard":"from typing import Any\n\ndef is_llm_json_object(raw_text: str) -> bool:\n    import json\n    try:\n        return isinstance(json.loads(raw_text.strip().strip('`')), dict)\n    except Exception:\n        return False","tryCatchPattern":"try:\n    data = _parse_llm_json(raw_text)\nexcept ValueError as e:\n    print(f'[SPEC] LLM did not return a JSON object: {e}')\n    data = {}  # or retry the agent call with a stricter prompt","preventionTips":["Pin the prompt in generate_golden_spec.md to require a top-level JSON object with an example.","Log raw_text before parsing so failures are reproducible.","Add a retry: if the first parse fails, re-prompt the agent asking it to return only a JSON object.","Add a unit test feeding known-bad LLM outputs (arrays, prose) through _parse_llm_json."],"tags":["llm","json-parsing","golden-spec","antigravity","evals","python"],"backgroundTag":null,"analyzedSha":"5024443c7217464a66e98f80d73172a26440bd8f","analyzedAt":"2026-08-12T06:01:53.711Z","schemaVersion":2},"datasetVersion":"2026-08-12T12:31:55.035Z"}