rohitg00/ai-engineering-from-scratch · error · ValueError

uncertainFallback is required

Error message

uncertainFallback is required

What it means

Error "uncertainFallback is required" thrown in rohitg00/ai-engineering-from-scratch.

Source

Thrown at certifications/claude/lessons/02-model-selection-and-token-economics/code/main.py:63

        if not str(case.get("routingSignal", "")).strip():
            errors.append(f"cases[{index}] needs an observable routingSignal")
        if case.get("riskClass") == "consequential" and case.get("humanReview") is not True:
            errors.append(f"cases[{index}] consequential work must have humanReview")
    comparison = benchmark.get("routingComparison")
    routed = sum(case.get("estimatedCostUnits", 0) for case in cases if isinstance(case, dict))
    if not isinstance(comparison, dict) or comparison.get("routedCostUnits") != routed:
        errors.append("routedCostUnits must equal the case total")
    elif comparison.get("allCapableCostUnits", 0) <= routed:
        errors.append("allCapableCostUnits must exceed routedCostUnits")
    if not isinstance(comparison, dict) or not str(comparison.get("uncertainFallback", "")).strip():
        errors.append("uncertainFallback is required")
    return errors


def summarize(benchmark: dict[str, Any]) -> dict[str, Any]:
    errors = validate_benchmark(benchmark)
    if errors:
        raise ValueError("; ".join(errors))
    lanes = Counter(case["chosenModel"] for case in benchmark["cases"])
    comparison = benchmark["routingComparison"]
    return {
        "caseCount": len(benchmark["cases"]),
        "lanes": dict(sorted(lanes.items())),
        "costSavedUnits": comparison["allCapableCostUnits"] - comparison["routedCostUnits"],
        "humanReviewCases": sum(case["humanReview"] for case in benchmark["cases"]),
    }


def validate_mode_trials(experiment: dict[str, Any]) -> list[str]:
    errors: list[str] = []
    if experiment.get("measurementStatus") != "illustrative-not-live-provider-runs":
        errors.append("mode trials must identify illustrative measurements")
    if not str(experiment.get("settingSemantics", "")).strip():
        errors.append("mode trials must explain normalized setting labels")
    verified_on = experiment.get("verifiedOn")
    if not _iso_date(verified_on):

View on GitHub (pinned to 39ea8a1c6d)

When it happens

Trigger: Thrown at certifications/claude/lessons/02-model-selection-and-token-economics/code/main.py:63 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of rohitg00/ai-engineering-from-scratch@39ea8a1c6d (2026-08-26). Data as JSON: /api/errors/a063f7f3aa54d598. Report an issue: GitHub.