google-gemini/gemini-cli · error · ValueError

Expected JSON object from LLM, but got {type(data).name}

Error message

Expected JSON object from LLM, but got {type(data).__name__}. Raw output:\n{raw_text}

What it means

This ValueError is raised in _parse_llm_json() when the Antigravity spec-generator LLM returns text that parses as valid JSON but is not a top-level JSON object (dict). After stripping markdown fences and applying a fallback unescape pass, if json.loads yields a list, string, number, bool, or None instead of a dict, the function rejects it. It guards the contract that the golden-spec agent must emit a structured object.

Source

Thrown at tools/caretaker-agent/evals/triage/helpers/generate_golden_spec.py:44

from google.antigravity import Agent, LocalAgentConfig
from google.antigravity.hooks.policy import deny

PROMPT_FILE = Path(__file__).parent / "generate_golden_spec.md"


def _parse_llm_json(raw_text: str) -> dict:
    """Strips markdown fences and parses LLM JSON with fallback unescaping."""
    clean = raw_text.strip()
    if clean.startswith("```"):
        clean = clean.split("\n", 1)[-1].rsplit("\n", 1)[0].strip()
    try:
        data = json.loads(clean, strict=False)
    except Exception:
        cleaned = re.sub(r'\\(?![/"bfnrtu]|u[0-9a-fA-F]{4})', r'\\\\', re.sub(r"(?<!\\)\\'", "'", clean))
        data = json.loads(cleaned, strict=False)

    if not isinstance(data, dict):
        raise ValueError(f"Expected JSON object from LLM, but got {type(data).__name__}. Raw output:\n{raw_text}")

    return data


def _load_system_instruction() -> str:
    """Loads prompt instructions from generate_golden_spec.md."""
    if not PROMPT_FILE.exists():
        raise FileNotFoundError(f"Required prompt file missing at: {PROMPT_FILE}")
    with open(PROMPT_FILE, "r", encoding="utf-8") as f:
        return f.read()


def generate_golden_spec(owner: str, repo: str, issue_number: int, issue_data: dict, pr_data: dict) -> dict:
    """
    Invokes the Antigravity SDK (google.antigravity) Agent using generate_golden_spec.md
    instructions to synthesize a clean, high-precision Workable Spec JSON and its rationale.
    Returns a dict with keys: 'workable_spec' and 'golden_spec_rationale'.
    """

View on GitHub (pinned to 5024443c72)

Solutions

Log or print raw_text before parsing to see exactly what the LLM returned and where it diverges from an object.
Update generate_golden_spec.md to explicitly require a top-level JSON object and include a conforming example.
If the model persistently returns a list, wrap the expectation: data = data[0] if isinstance(data, list) and data and isinstance(data[0], dict) else data, then re-validate.
Verify extract_final_output(resolved_chunks) returns the complete final agent message and not a truncated or multi-part stream.

Example fix

# before: model returns ["workable_spec", {...}]
# fix prompt in generate_golden_spec.md to show:
# Respond with a single JSON object, e.g.:
# {"workable_spec": {...}, "golden_spec_rationale": "..."}

Defensive patterns

Strategy: type-guard

Validate before calling

import json, re

def safe_parse_llm_json(raw_text: str) -> dict:
    clean = raw_text.strip()
    if clean.startswith('```'):
        clean = clean.split('\n', 1)[-1].rsplit('\n', 1)[0].strip()
    try:
        data = json.loads(clean, strict=False)
    except Exception:
        cleaned = re.sub(r'\\(?![/"bfnrtu]|u[0-9a-fA-F]{4})', r'\\\\', re.sub(r"(?<!\\\\)\\'", "'", clean))
        data = json.loads(cleaned, strict=False)
    if isinstance(data, list) and data and isinstance(data[0], dict):
        return data[0]
    if not isinstance(data, dict):
        raise ValueError(f'Expected JSON object, got {type(data).__name__}')
    return data

Type guard

from typing import Any

def is_llm_json_object(raw_text: str) -> bool:
    import json
    try:
        return isinstance(json.loads(raw_text.strip().strip('`')), dict)
    except Exception:
        return False

Try / catch

try:
    data = _parse_llm_json(raw_text)
except ValueError as e:
    print(f'[SPEC] LLM did not return a JSON object: {e}')
    data = {}  # or retry the agent call with a stricter prompt

Prevention

Pin the prompt in generate_golden_spec.md to require a top-level JSON object with an example.
Log raw_text before parsing so failures are reproducible.
Add a retry: if the first parse fails, re-prompt the agent asking it to return only a JSON object.
Add a unit test feeding known-bad LLM outputs (arrays, prose) through _parse_llm_json.

When it happens

Trigger: The LLM emits a JSON array of items, a bare quoted string, or a number instead of an object. The agent wraps its answer in extra prose so the fence-stripping logic extracts the wrong segment. A model or SDK version change causes the response to be a top-level scalar.

Common situations: A prompt update removed the instruction to return a JSON object. The model returns a list because the prompt example used an array shape. The markdown fence stripper in _parse_llm_json mis-handles nested code blocks or a leading language tag, leaving non-JSON content that happens to parse as a non-dict type. extract_final_output concatenates chunks in an order that produces a fragment.

Related errors

AI-assisted analysis of google-gemini/gemini-cli@5024443c72 (2026-08-12). Data as JSON: /api/errors/3a117c1c5fe877ed. Report an issue: GitHub.