rohitg00/ai-engineering-from-scratch · error · ValueError

score for {r.model_id}/{r.task_id} not in [0,1]: {r.score}

Error message

score for {r.model_id}/{r.task_id} not in [0,1]: {r.score}

What it means

Error "score for {r.model_id}/{r.task_id} not in [0,1]: {r.score}" thrown in rohitg00/ai-engineering-from-scratch.

Source

Thrown at phases/19-capstone-projects/74-leaderboard-aggregation/code/main.py:79

    ci_hi: float
    significant: bool

    def to_dict(self) -> dict:
        return {
            "model_a": self.model_a,
            "model_b": self.model_b,
            "diff_mean": self.diff_mean,
            "ci_lo": self.ci_lo,
            "ci_hi": self.ci_hi,
            "significant": self.significant,
        }


def _validate_runs(runs: Sequence[EvalRun]) -> None:
    seen: set[tuple[str, str]] = set()
    for r in runs:
        if not (0.0 <= r.score <= 1.0):
            raise ValueError(f"score for {r.model_id}/{r.task_id} not in [0,1]: {r.score}")
        key = (r.model_id, r.task_id)
        if key in seen:
            raise ValueError(f"duplicate run for {r.model_id}/{r.task_id}")
        seen.add(key)


def _by_model(runs: Sequence[EvalRun]) -> dict[str, list[EvalRun]]:
    out: dict[str, list[EvalRun]] = defaultdict(list)
    for r in runs:
        out[r.model_id].append(r)
    return dict(out)


def _by_task(runs: Sequence[EvalRun]) -> dict[str, list[EvalRun]]:
    out: dict[str, list[EvalRun]] = defaultdict(list)
    for r in runs:
        out[r.task_id].append(r)
    return dict(out)

View on GitHub (pinned to 39ea8a1c6d)

When it happens

Trigger: Thrown at phases/19-capstone-projects/74-leaderboard-aggregation/code/main.py:79 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of rohitg00/ai-engineering-from-scratch@39ea8a1c6d (2026-08-26). Data as JSON: /api/errors/e29a64116b4eefe8. Report an issue: GitHub.