{"record":{"id":"5588dcf5cdfdce7b","repo":"shareAI-lab/learn-claude-code","slug":"goal-evaluator-response-requires-boolean-ok","errorCode":null,"errorMessage":"goal evaluator response requires boolean 'ok'","messagePattern":"goal evaluator response requires boolean 'ok'","errorType":"validation","errorClass":"GoalError","httpStatus":null,"severity":"error","filePath":"s17_goal_loop/code.py","lineNumber":187,"sourceCode":"\n\ndef _parse_json_object(text: str) -> dict[str, Any]:\n    stripped = text.strip()\n    if stripped.startswith(\"```\"):\n        lines = stripped.splitlines()\n        if lines and lines[0].startswith(\"```\"):\n            lines = lines[1:]\n        if lines and lines[-1].strip() == \"```\":\n            lines = lines[:-1]\n        stripped = \"\\n\".join(lines).strip()\n    try:\n        value = json.loads(stripped)\n    except json.JSONDecodeError as error:\n        raise GoalError(\"goal evaluator returned invalid JSON\") from error\n    if not isinstance(value, dict):\n        raise GoalError(\"goal evaluator must return a JSON object\")\n    if not isinstance(value.get(\"ok\"), bool):\n        raise GoalError(\"goal evaluator response requires boolean 'ok'\")\n    if not isinstance(value.get(\"reason\"), str) or not value[\"reason\"].strip():\n        raise GoalError(\"goal evaluator response requires non-empty 'reason'\")\n    impossible = value.get(\"impossible\", False)\n    if not isinstance(impossible, bool):\n        raise GoalError(\"goal evaluator 'impossible' must be boolean\")\n    if value[\"ok\"] and impossible:\n        raise GoalError(\n            \"goal evaluator cannot return both ok and impossible\"\n        )\n    return {\n        \"ok\": value[\"ok\"],\n        \"reason\": value[\"reason\"].strip(),\n        \"impossible\": impossible,\n    }\n\n\nclass PromptGoalEvaluator:\n    \"\"\"A separate, tool-free model that judges the transcript.\"\"\"","sourceCodeStart":169,"sourceCodeEnd":205,"githubUrl":"https://github.com/shareAI-lab/learn-claude-code/blob/985456f4adea6f4df8fbad4112245dbd97444eae/s17_goal_loop/code.py#L169-L205","documentation":"The parsed evaluator object must contain 'ok' as an actual bool; missing 'ok', a truthy 1/\"true\", or null raises GoalError(\"goal evaluator response requires boolean 'ok'\"). Python truthiness is deliberately not accepted — the allow/block decision must be unambiguous.","triggerScenarios":"Evaluator JSON like {\"reason\": \"...\"} (no ok), {\"ok\": \"true\"}, or {\"ok\": 1} — the value is absent or not of boolean type.","commonSituations":"Prompts that omit the exact field names; models emitting 1/0 or \"true\"/\"false\" strings; schema not constraining ok to type:boolean.","solutions":["Constrain the evaluator response schema: {\"ok\": {\"type\": \"boolean\"}} and mark it required.","Add a one-shot example in the evaluator prompt showing true/false literals.","For custom evaluators, coerce to bool in code before returning the JSON text."],"exampleFix":"# before: {\"ok\": 1, \"reason\": \"done\"}\n\n# after: {\"ok\": true, \"reason\": \"done\"}","handlingStrategy":"type-guard","validationCode":"import json\nobj = json.loads(evaluator_text)\nok = obj.get(\"ok\")\nassert ok is True or ok is False, \"'ok' must be a JSON boolean literal\"","typeGuard":"def ok_is_bool(value: object) -> bool:\n    return isinstance(value, dict) and isinstance(value.get(\"ok\"), bool)","tryCatchPattern":"try:\n    ev = _parse_json_object(text)\nexcept GoalError as e:\n    if \"boolean 'ok'\" in str(e):\n        obj = json.loads(text)\n        obj[\"ok\"] = bool(obj.get(\"ok\"))  # only if 1/0 semantics are acceptable to you\n        ev = _parse_json_object(json.dumps(obj))\n    else:\n        raise","preventionTips":["Constrain 'ok' to type:boolean in the evaluator's response schema or JSON mode.","Include a literal example ({\"ok\": true, ...}) in the evaluator prompt.","For custom evaluators, emit only true/false literals, never 1/0 or strings."],"tags":["goal-loop","evaluator","json","type-validation","llm"],"backgroundTag":null,"analyzedSha":"985456f4adea6f4df8fbad4112245dbd97444eae","analyzedAt":"2026-08-14T22:02:26.028Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}