shareAI-lab/learn-claude-code · error · GoalError

goal evaluator response requires non-empty 'reason'

Error message

goal evaluator response requires non-empty 'reason'

What it means

_parse_json_object requires a 'reason' that is a string and non-empty after strip(); otherwise GoalError("goal evaluator response requires non-empty 'reason'"). Every allow/block decision must carry a human-readable justification, which the loop feeds back to the worker model.

Source

Thrown at s17_goal_loop/code.py:189

def _parse_json_object(text: str) -> dict[str, Any]:
    stripped = text.strip()
    if stripped.startswith("```"):
        lines = stripped.splitlines()
        if lines and lines[0].startswith("```"):
            lines = lines[1:]
        if lines and lines[-1].strip() == "```":
            lines = lines[:-1]
        stripped = "\n".join(lines).strip()
    try:
        value = json.loads(stripped)
    except json.JSONDecodeError as error:
        raise GoalError("goal evaluator returned invalid JSON") from error
    if not isinstance(value, dict):
        raise GoalError("goal evaluator must return a JSON object")
    if not isinstance(value.get("ok"), bool):
        raise GoalError("goal evaluator response requires boolean 'ok'")
    if not isinstance(value.get("reason"), str) or not value["reason"].strip():
        raise GoalError("goal evaluator response requires non-empty 'reason'")
    impossible = value.get("impossible", False)
    if not isinstance(impossible, bool):
        raise GoalError("goal evaluator 'impossible' must be boolean")
    if value["ok"] and impossible:
        raise GoalError(
            "goal evaluator cannot return both ok and impossible"
        )
    return {
        "ok": value["ok"],
        "reason": value["reason"].strip(),
        "impossible": impossible,
    }


class PromptGoalEvaluator:
    """A separate, tool-free model that judges the transcript."""

    def __init__(

View on GitHub (pinned to 985456f4ad)

Solutions

1. State in the evaluator prompt that a non-empty 'reason' string is required in every response, including successes.
2. Force the field in the response schema (required + type:string + minLength).
3. For custom evaluators, always populate reason (e.g. default to a summary line) before returning.

Example fix

# before: {"ok": true}

# after: {"ok": true, "reason": "pytest exits 0; goal satisfied"}

import json
obj = json.loads(evaluator_text)
reason = obj.get("reason")
assert isinstance(reason, str) and reason.strip(), "'reason' must be a non-empty string"

def reason_is_valid(value: object) -> bool:
    return isinstance(value, dict) and isinstance(value.get("reason"), str) and bool(value["reason"].strip())

try:
    ev = _parse_json_object(text)
except GoalError as e:
    if "non-empty 'reason'" in str(e):
        obj = json.loads(text)
        obj["reason"] = obj.get("reason") or "evaluator gave no reason"
        ev = _parse_json_object(json.dumps(obj))
    else:
        raise

When it happens

Trigger: Evaluator JSON with reason missing, reason: "", reason: " ", or reason as a non-string (e.g. a list of strings).

Common situations: Model economizing tokens and omitting reason when ok=true; prompt failing to say reason is mandatory; reason modeled as an array of bullet points.

Related errors

• goal evaluator must return a JSON object
• goal evaluator returned invalid JSON
• goal evaluator response requires boolean 'ok'
• goal evaluator 'impossible' must be boolean
• goal evaluator cannot return both ok and impossible

AI-assisted analysis of shareAI-lab/learn-claude-code@985456f4ad (2026-08-14). Data as JSON: /api/errors/e848f85cf203a0ff. Report an issue: GitHub.