rohitg00/ai-engineering-from-scratch · error · ValueError

release gate root must be an object

Error message

release gate root must be an object

What it means

Raised in load_release_gate when the parsed release-gate JSON's root is not an object. The gate validator reads keyed members (severeCasePassRate, severeCases, evaluationSurfaces, trace fields), so a non-object root means the gate file is structurally invalid and validation cannot proceed.

Solutions

1. Fix the release-gate JSON so its root is an object with the required keyed sections
2. Regenerate the gate file from the eval harness instead of hand-editing
3. Catch ValueError in the tests to assert the gate fixture fails structurally when corrupted

When it happens

Trigger: Thrown at certifications/claude/lessons/14-evals-testing-debugging-and-observability/code/main.py:133 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of rohitg00/ai-engineering-from-scratch@39ea8a1c6d (2026-08-26). Data as JSON: /api/errors/0122c0878ac28f3c. Report an issue: GitHub.

Appendix: source

Thrown at certifications/claude/lessons/14-evals-testing-debugging-and-observability/code/main.py:133

        errors.append("severeCasePassRate must be 1.0")
    if not isinstance(gate.get("severeCases"), list) or len(gate["severeCases"]) < 3:
        errors.append("at least three severeCases are required")
    required_surfaces = {"output", "trajectory", "final-state", "safety", "operational-budget"}
    if not required_surfaces <= set(gate.get("evaluationSurfaces", [])):
        errors.append("all five evaluation surfaces are required")
    required_trace = {"trace_id", "type", "model_version", "prompt_version", "policy", "latency_ms", "result_class"}
    if not required_trace <= set(gate.get("requiredTraceFields", [])):
        errors.append("required trace fields are incomplete")
    required_failures = {"transport_timeout", "protocol_error", "contract_parse_error", "policy_denial", "final_state_failure"}
    if not required_failures <= set(gate.get("failureClasses", [])):
        errors.append("failure classes are incomplete")
    return errors


def load_release_gate(path: str) -> dict[str, Any]:
    value = json.loads(Path(path).read_text(encoding="utf-8"))
    if not isinstance(value, dict):
        raise ValueError("release gate root must be an object")
    return value


def _elapsed_ms(started: float) -> float:
    return (time.perf_counter() - started) * 1000


def demo_agent(prompt: str) -> AgentRun:
    if "missing" in prompt:
        return AgentRun("I cannot verify that order.", (), {"escalated": True}, ({"type": "decision", "reason": "missing_id"},))
    return AgentRun("Order A-17 is ready.", ("lookup_order",), {"escalated": False}, ({"type": "tool_call", "name": "lookup_order"},))


def demo() -> dict[str, Any]:
    cases = [
        EvalCase("known", "Find A-17", ("ready",), (), ("lookup_order",), {"escalated": False}),
        EvalCase("missing", "The order id is missing", ("cannot verify",), ("ready",), (), {"escalated": True}),
    ]

View on GitHub (pinned to 39ea8a1c6d)