rohitg00/ai-engineering-from-scratch · error · ValueError

max_norm must be positive

Error message

max_norm must be positive

What it means

Error "max_norm must be positive" thrown in rohitg00/ai-engineering-from-scratch.

Source

Thrown at phases/19-capstone-projects/45-gradient-clipping-amp/code/main.py:112

            continue
        grad = param.grad.detach()
        squared_sum += float(grad.pow(2).sum().item())
    return math.sqrt(squared_sum)


def clip_global_l2_norm(
    parameters: list[torch.nn.Parameter],
    max_norm: float,
) -> tuple[float, float]:
    """Clip gradients in place to max_norm and return (pre_clip, post_clip).

    Returns (pre_clip, post_clip). When pre_clip <= max_norm the gradients are
    untouched and post_clip == pre_clip. When pre_clip > max_norm the gradients
    are scaled by max_norm / pre_clip and post_clip == max_norm.
    """

    if max_norm <= 0:
        raise ValueError("max_norm must be positive")
    pre_clip = compute_global_l2_norm(parameters)
    if not math.isfinite(pre_clip):
        return pre_clip, pre_clip
    if pre_clip <= max_norm:
        return pre_clip, pre_clip
    scale = max_norm / (pre_clip + 1e-12)
    for param in parameters:
        if param.grad is not None:
            param.grad.detach().mul_(scale)
    return pre_clip, max_norm


class AmpTrainState:
    """Training step with mixed precision and gradient clipping.

    Wires together a model, an AdamW optimizer, a GradScaler, and an autocast
    device. Exposes step(inputs, targets) which:

View on GitHub (pinned to 39ea8a1c6d)

When it happens

Trigger: Thrown at phases/19-capstone-projects/45-gradient-clipping-amp/code/main.py:112 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of rohitg00/ai-engineering-from-scratch@39ea8a1c6d (2026-08-26). Data as JSON: /api/errors/2b0f554ca5d2c321. Report an issue: GitHub.