{"record":{"id":"e848f85cf203a0ff","repo":"shareAI-lab/learn-claude-code","slug":"goal-evaluator-response-requires-non-empty-reason","errorCode":null,"errorMessage":"goal evaluator response requires non-empty 'reason'","messagePattern":"goal evaluator response requires non-empty 'reason'","errorType":"validation","errorClass":"GoalError","httpStatus":null,"severity":"error","filePath":"s17_goal_loop/code.py","lineNumber":189,"sourceCode":"def _parse_json_object(text: str) -> dict[str, Any]:\n    stripped = text.strip()\n    if stripped.startswith(\"```\"):\n        lines = stripped.splitlines()\n        if lines and lines[0].startswith(\"```\"):\n            lines = lines[1:]\n        if lines and lines[-1].strip() == \"```\":\n            lines = lines[:-1]\n        stripped = \"\\n\".join(lines).strip()\n    try:\n        value = json.loads(stripped)\n    except json.JSONDecodeError as error:\n        raise GoalError(\"goal evaluator returned invalid JSON\") from error\n    if not isinstance(value, dict):\n        raise GoalError(\"goal evaluator must return a JSON object\")\n    if not isinstance(value.get(\"ok\"), bool):\n        raise GoalError(\"goal evaluator response requires boolean 'ok'\")\n    if not isinstance(value.get(\"reason\"), str) or not value[\"reason\"].strip():\n        raise GoalError(\"goal evaluator response requires non-empty 'reason'\")\n    impossible = value.get(\"impossible\", False)\n    if not isinstance(impossible, bool):\n        raise GoalError(\"goal evaluator 'impossible' must be boolean\")\n    if value[\"ok\"] and impossible:\n        raise GoalError(\n            \"goal evaluator cannot return both ok and impossible\"\n        )\n    return {\n        \"ok\": value[\"ok\"],\n        \"reason\": value[\"reason\"].strip(),\n        \"impossible\": impossible,\n    }\n\n\nclass PromptGoalEvaluator:\n    \"\"\"A separate, tool-free model that judges the transcript.\"\"\"\n\n    def __init__(","sourceCodeStart":171,"sourceCodeEnd":207,"githubUrl":"https://github.com/shareAI-lab/learn-claude-code/blob/985456f4adea6f4df8fbad4112245dbd97444eae/s17_goal_loop/code.py#L171-L207","documentation":"_parse_json_object requires a 'reason' that is a string and non-empty after strip(); otherwise GoalError(\"goal evaluator response requires non-empty 'reason'\"). Every allow/block decision must carry a human-readable justification, which the loop feeds back to the worker model.","triggerScenarios":"Evaluator JSON with reason missing, reason: \"\", reason: \"   \", or reason as a non-string (e.g. a list of strings).","commonSituations":"Model economizing tokens and omitting reason when ok=true; prompt failing to say reason is mandatory; reason modeled as an array of bullet points.","solutions":["State in the evaluator prompt that a non-empty 'reason' string is required in every response, including successes.","Force the field in the response schema (required + type:string + minLength).","For custom evaluators, always populate reason (e.g. default to a summary line) before returning."],"exampleFix":"# before: {\"ok\": true}\n\n# after: {\"ok\": true, \"reason\": \"pytest exits 0; goal satisfied\"}","handlingStrategy":"validation","validationCode":"import json\nobj = json.loads(evaluator_text)\nreason = obj.get(\"reason\")\nassert isinstance(reason, str) and reason.strip(), \"'reason' must be a non-empty string\"","typeGuard":"def reason_is_valid(value: object) -> bool:\n    return isinstance(value, dict) and isinstance(value.get(\"reason\"), str) and bool(value[\"reason\"].strip())","tryCatchPattern":"try:\n    ev = _parse_json_object(text)\nexcept GoalError as e:\n    if \"non-empty 'reason'\" in str(e):\n        obj = json.loads(text)\n        obj[\"reason\"] = obj.get(\"reason\") or \"evaluator gave no reason\"\n        ev = _parse_json_object(json.dumps(obj))\n    else:\n        raise","preventionTips":["State in the evaluator prompt that reason is mandatory on every response, including ok:true.","Require reason (string, minLength 1) in the response schema.","In custom evaluators, default reason to a one-line summary when the model omits it."],"tags":["goal-loop","evaluator","json","validation","llm"],"backgroundTag":null,"analyzedSha":"985456f4adea6f4df8fbad4112245dbd97444eae","analyzedAt":"2026-08-14T22:02:26.028Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}