shareAI-lab/learn-claude-code · error · GoalError

goal evaluator response requires boolean 'ok'

Error message

goal evaluator response requires boolean 'ok'

What it means

The parsed evaluator object must contain 'ok' as an actual bool; missing 'ok', a truthy 1/"true", or null raises GoalError("goal evaluator response requires boolean 'ok'"). Python truthiness is deliberately not accepted — the allow/block decision must be unambiguous.

Source

Thrown at s17_goal_loop/code.py:187

def _parse_json_object(text: str) -> dict[str, Any]:
    stripped = text.strip()
    if stripped.startswith("```"):
        lines = stripped.splitlines()
        if lines and lines[0].startswith("```"):
            lines = lines[1:]
        if lines and lines[-1].strip() == "```":
            lines = lines[:-1]
        stripped = "\n".join(lines).strip()
    try:
        value = json.loads(stripped)
    except json.JSONDecodeError as error:
        raise GoalError("goal evaluator returned invalid JSON") from error
    if not isinstance(value, dict):
        raise GoalError("goal evaluator must return a JSON object")
    if not isinstance(value.get("ok"), bool):
        raise GoalError("goal evaluator response requires boolean 'ok'")
    if not isinstance(value.get("reason"), str) or not value["reason"].strip():
        raise GoalError("goal evaluator response requires non-empty 'reason'")
    impossible = value.get("impossible", False)
    if not isinstance(impossible, bool):
        raise GoalError("goal evaluator 'impossible' must be boolean")
    if value["ok"] and impossible:
        raise GoalError(
            "goal evaluator cannot return both ok and impossible"
        )
    return {
        "ok": value["ok"],
        "reason": value["reason"].strip(),
        "impossible": impossible,
    }


class PromptGoalEvaluator:
    """A separate, tool-free model that judges the transcript."""

View on GitHub (pinned to 985456f4ad)

Solutions

1. Constrain the evaluator response schema: {"ok": {"type": "boolean"}} and mark it required.
2. Add a one-shot example in the evaluator prompt showing true/false literals.
3. For custom evaluators, coerce to bool in code before returning the JSON text.

Example fix

# before: {"ok": 1, "reason": "done"}

# after: {"ok": true, "reason": "done"}

import json
obj = json.loads(evaluator_text)
ok = obj.get("ok")
assert ok is True or ok is False, "'ok' must be a JSON boolean literal"

def ok_is_bool(value: object) -> bool:
    return isinstance(value, dict) and isinstance(value.get("ok"), bool)

try:
    ev = _parse_json_object(text)
except GoalError as e:
    if "boolean 'ok'" in str(e):
        obj = json.loads(text)
        obj["ok"] = bool(obj.get("ok"))  # only if 1/0 semantics are acceptable to you
        ev = _parse_json_object(json.dumps(obj))
    else:
        raise

When it happens

Trigger: Evaluator JSON like {"reason": "..."} (no ok), {"ok": "true"}, or {"ok": 1} — the value is absent or not of boolean type.

Common situations: Prompts that omit the exact field names; models emitting 1/0 or "true"/"false" strings; schema not constraining ok to type:boolean.

Related errors

• goal evaluator 'impossible' must be boolean
• goal evaluator returned invalid JSON
• goal evaluator must return a JSON object
• goal evaluator response requires non-empty 'reason'
• goal evaluator cannot return both ok and impossible

AI-assisted analysis of shareAI-lab/learn-claude-code@985456f4ad (2026-08-14). Data as JSON: /api/errors/5588dcf5cdfdce7b. Report an issue: GitHub.