huggingface/pytorch-image-models · error · ValueError

Eps must be non-negative

Error message

Eps must be non-negative

What it means

The numerical-stability epsilon in MADGRAD's denominator must be non-negative; a negative eps is rejected because it can invalidate the sqrt/normalization steps of the update.

Source

Thrown at timm/optim/madgrad.py:71

    """

    def __init__(
            self,
            params: _params_t,
            lr: float = 1e-2,
            momentum: float = 0.9,
            weight_decay: float = 0,
            eps: float = 1e-6,
            decoupled_decay: bool = False,
    ):
        if momentum < 0 or momentum >= 1:
            raise ValueError(f"Momentum {momentum} must be in the range [0,1]")
        if lr <= 0:
            raise ValueError(f"Learning rate {lr} must be positive")
        if weight_decay < 0:
            raise ValueError(f"Weight decay {weight_decay} must be non-negative")
        if eps < 0:
            raise ValueError(f"Eps must be non-negative")

        defaults = dict(
            lr=lr,
            eps=eps,
            momentum=momentum,
            weight_decay=weight_decay,
            decoupled_decay=decoupled_decay,
        )
        super().__init__(params, defaults)

    @property
    def supports_memory_efficient_fp16(self) -> bool:
        return False

    @property
    def supports_flat_params(self) -> bool:
        return True

View on GitHub (pinned to 9a5261e31b)

Solutions

1. Use a positive eps (default 1e-6)
2. Fix the sign in the config/sweep generating eps

Example fix

# before
opt = MADGRAD(model.parameters(), eps=-1e-6)
# after
opt = MADGRAD(model.parameters(), eps=1e-6)

assert cfg.eps >= 0, 'eps must be non-negative'

def is_valid_eps(e: float) -> bool:
    return isinstance(e, (int, float)) and e >= 0

When it happens

Trigger: Passing eps<0 (e.g. eps=-1e-6, a sign typo) to the MADGRAD constructor.

Common situations: Typos in config files; programmatically derived eps values with a wrong sign; copy-paste from a config that used a negative offset for something else.

Related errors

• Momentum {momentum} must be in the range [0,1]
• Learning rate {lr} must be positive
• Weight decay {weight_decay} must be non-negative
• Invalid epsilon value: {}
• momentum != 0 is not compatible with sparse gradients

AI-assisted analysis of huggingface/pytorch-image-models@9a5261e31b (2026-08-27). Data as JSON: /api/errors/403dfa8a439aa8f5. Report an issue: GitHub.