shareAI-lab/learn-claude-code · error · GoalError

goal evaluator must return a JSON object

Error message

goal evaluator must return a JSON object

What it means

After successful json.loads, _parse_json_object requires the top-level value to be a dict; anything else (array, string, number) raises GoalError('goal evaluator must return a JSON object'). The downstream contract reads keys ok/reason/impossible, so a non-object reply cannot be interpreted.

Source

Thrown at s17_goal_loop/code.py:185

        size += item_size
    return "\n\n".join(reversed(selected))


def _parse_json_object(text: str) -> dict[str, Any]:
    stripped = text.strip()
    if stripped.startswith("```"):
        lines = stripped.splitlines()
        if lines and lines[0].startswith("```"):
            lines = lines[1:]
        if lines and lines[-1].strip() == "```":
            lines = lines[:-1]
        stripped = "\n".join(lines).strip()
    try:
        value = json.loads(stripped)
    except json.JSONDecodeError as error:
        raise GoalError("goal evaluator returned invalid JSON") from error
    if not isinstance(value, dict):
        raise GoalError("goal evaluator must return a JSON object")
    if not isinstance(value.get("ok"), bool):
        raise GoalError("goal evaluator response requires boolean 'ok'")
    if not isinstance(value.get("reason"), str) or not value["reason"].strip():
        raise GoalError("goal evaluator response requires non-empty 'reason'")
    impossible = value.get("impossible", False)
    if not isinstance(impossible, bool):
        raise GoalError("goal evaluator 'impossible' must be boolean")
    if value["ok"] and impossible:
        raise GoalError(
            "goal evaluator cannot return both ok and impossible"
        )
    return {
        "ok": value["ok"],
        "reason": value["reason"].strip(),
        "impossible": impossible,
    }

View on GitHub (pinned to 985456f4ad)

Solutions

1. Fix the evaluator prompt/schema to demand a top-level object with ok/reason/impossible.
2. If the evaluator can only produce arrays, wrap its output into an object before it reaches _parse_json_object.
3. Return the literal object, not its serialized string, from custom evaluators.

Example fix

# before: evaluator text = '[{"ok": true}]'  (array at top level)

# after: evaluator text = '{"ok": true, "reason": "tests pass", "impossible": false}'

import json
def evaluator_returns_object(text: str) -> bool:
    try:
        return isinstance(json.loads(text.strip().strip('`')), dict)
    except json.JSONDecodeError:
        return False

def is_evaluator_object(text: str) -> bool:
    stripped = text.strip()
    if stripped.startswith('```'):
        stripped = stripped.splitlines()[1]
    try:
        return isinstance(json.loads(stripped), dict)
    except json.JSONDecodeError:
        return False

try:
    ev = _parse_json_object(text)
except GoalError as e:
    if "must return a JSON object" in str(e):
        ev = _parse_json_object('{"ok": false, "reason": ' + json.dumps(text) + '}')  # treat as block-with-reason
    else:
        raise

When it happens

Trigger: Evaluator returns a JSON array of findings or a quoted JSON string (double-encoded) instead of a bare object.

Common situations: Prompt asking the model to 'list reasons' (elicits an array); JSON.stringify'd strings passed through; schemas declaring type:array for the evaluator response.

Related errors

• goal evaluator response requires non-empty 'reason'
• goal evaluator returned invalid JSON
• goal evaluator response requires boolean 'ok'
• goal evaluator 'impossible' must be boolean
• goal evaluator cannot return both ok and impossible

AI-assisted analysis of shareAI-lab/learn-claude-code@985456f4ad (2026-08-14). Data as JSON: /api/errors/d22148bc0d69a806. Report an issue: GitHub.