huggingface/pytorch-image-models · error · ValueError

Momentum {momentum} must be in the range [0,1]

Error message

Momentum {momentum} must be in the range [0,1]

What it means

MADGRAD optimizer validates that its momentum hyperparameter lies in [0, 1). Momentum of 1 or above, or a negative value, makes the momentum buffer diverge from the parameter trajectory, so the constructor rejects it immediately.

Source

Thrown at timm/optim/madgrad.py:65

        momentum (float):
            Momentum value in  the range [0,1) (default: 0.9).
        weight_decay (float):
            Weight decay, i.e. a L2 penalty (default: 0).
        eps (float):
            Term added to the denominator outside of the root operation to improve numerical stability. (default: 1e-6).
    """

    def __init__(
            self,
            params: _params_t,
            lr: float = 1e-2,
            momentum: float = 0.9,
            weight_decay: float = 0,
            eps: float = 1e-6,
            decoupled_decay: bool = False,
    ):
        if momentum < 0 or momentum >= 1:
            raise ValueError(f"Momentum {momentum} must be in the range [0,1]")
        if lr <= 0:
            raise ValueError(f"Learning rate {lr} must be positive")
        if weight_decay < 0:
            raise ValueError(f"Weight decay {weight_decay} must be non-negative")
        if eps < 0:
            raise ValueError(f"Eps must be non-negative")

        defaults = dict(
            lr=lr,
            eps=eps,
            momentum=momentum,
            weight_decay=weight_decay,
            decoupled_decay=decoupled_decay,
        )
        super().__init__(params, defaults)

    @property
    def supports_memory_efficient_fp16(self) -> bool:

View on GitHub (pinned to 9a5261e31b)

Solutions

1. Set momentum to a value in [0, 1), e.g. the default 0.9
2. If you copied momentum from another optimizer's config, re-tune it for MADGRAD (0.9 or 0.95 are typical)
3. Add bounds validation in your hyperparameter sweep/config loader

Example fix

// before
opt = MADGRAD(model.parameters(), lr=1e-3, momentum=1.0)
// after
opt = MADGRAD(model.parameters(), lr=1e-3, momentum=0.9)

assert 0 <= cfg.momentum < 1, f'momentum {cfg.momentum} out of [0,1)'

def is_valid_momentum(m: float) -> bool:
    return isinstance(m, (int, float)) and 0 <= m < 1

When it happens

Trigger: Constructing madgrad.MADGRAD(params, momentum=1.0) or momentum=-0.1, or passing a momentum value sourced from a config/CLI without bounds checking. Note momentum=1 is rejected (range is [0,1) exclusive at the top).

Common situations: Copying momentum=0.999 or 1.0 settings from an Adam/SGD config where values close to 1 are common; sweeping momentum values without excluding the upper bound.

Related errors

• Learning rate {lr} must be positive
• Weight decay {weight_decay} must be non-negative
• Eps must be non-negative
• momentum != 0 is not compatible with sparse gradients
• weight_decay option is not compatible with sparse gradients

AI-assisted analysis of huggingface/pytorch-image-models@9a5261e31b (2026-08-27). Data as JSON: /api/errors/8d2f030ec3eb3491. Report an issue: GitHub.