rohitg00/ai-engineering-from-scratch · error · ValueError

evaluation plan requires a non-empty cases list

Error message

evaluation plan requires a non-empty cases list

What it means

Error "evaluation plan requires a non-empty cases list" thrown in rohitg00/ai-engineering-from-scratch.

Source

Thrown at certifications/claude/lessons/30-developer-application-capstone/code/main.py:222

    issues: list[str] = []
    required = {"status": str, "answer": str, "order_id": (str, type(None)), "escalated": bool}
    for name, expected in required.items():
        if name not in value:
            issues.append(f"missing {name}")
        elif not isinstance(value[name], expected):
            issues.append(f"invalid type for {name}")
    unexpected = set(value) - set(required)
    if unexpected:
        issues.append(f"unexpected fields: {sorted(unexpected)}")
    return issues


def load_eval_plan(path: Path | str | None = None) -> dict[str, Any]:
    source = Path(path) if path is not None else DEFAULT_EVAL_PLAN
    with source.open("r", encoding="utf-8") as handle:
        plan = json.load(handle)
    if not isinstance(plan, dict) or not isinstance(plan.get("cases"), list) or not plan["cases"]:
        raise ValueError("evaluation plan requires a non-empty cases list")
    if not isinstance(plan.get("releaseGates"), dict):
        raise ValueError("evaluation plan requires releaseGates")
    return plan


def _default_agent_factory(fixture: dict[str, str]) -> Callable[[str], SupportResult]:
    return SupportAgent(ToolRegistry(fixture)).run


def _trace_tools(trace: tuple[TraceEvent, ...]) -> list[str]:
    return [
        str(event.detail["tool"])
        for event in trace
        if event.type == "tool_result" and isinstance(event.detail.get("tool"), str)
    ]


def _trace_effects(trace: tuple[TraceEvent, ...]) -> set[str]:

View on GitHub (pinned to 39ea8a1c6d)

When it happens

Trigger: Thrown at certifications/claude/lessons/30-developer-application-capstone/code/main.py:222 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of rohitg00/ai-engineering-from-scratch@39ea8a1c6d (2026-08-26). Data as JSON: /api/errors/3e4e8194fccc66a9. Report an issue: GitHub.