rohitg00/ai-engineering-from-scratch · error · ValueError

unknown metric name: {name}

Error message

unknown metric name: {name}

What it means

Error "unknown metric name: {name}" thrown in rohitg00/ai-engineering-from-scratch.

Source

Thrown at phases/19-capstone-projects/41-eval-pipeline/code/main.py:636

    "exact_match": 0.3,
    "token_f1": 0.3,
    "judge": 0.2,
}


def normalise_metric(name: str, value: float) -> float:
    """Map a raw metric onto [0, 1]."""
    if name == "perplexity":
        if value <= 0 or math.isinf(value) or math.isnan(value):
            return 0.0
        return 1.0 / (1.0 + math.log(max(value, 1.0)))
    if name == "exact_match":
        return max(0.0, min(1.0, float(value)))
    if name == "token_f1":
        return max(0.0, min(1.0, float(value)))
    if name == "judge":
        return max(0.0, min(1.0, float(value) / 5.0))
    raise ValueError(f"unknown metric name: {name}")


@dataclass
class FinalReport:
    per_eval: Dict[str, float]
    normalised: Dict[str, float]
    weights: Dict[str, float]
    aggregate: float
    details: Dict[str, EvalResult] = field(default_factory=dict)

    def to_dict(self) -> Dict[str, Any]:
        return {
            "per_eval": self.per_eval,
            "normalised": self.normalised,
            "weights": self.weights,
            "aggregate": self.aggregate,
            "details": {k: v.to_dict() for k, v in self.details.items()},
        }

View on GitHub (pinned to 39ea8a1c6d)

When it happens

Trigger: Thrown at phases/19-capstone-projects/41-eval-pipeline/code/main.py:636 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of rohitg00/ai-engineering-from-scratch@39ea8a1c6d (2026-08-26). Data as JSON: /api/errors/916b3e7f04466eea. Report an issue: GitHub.