{"record":{"id":"8d2f030ec3eb3491","repo":"huggingface/pytorch-image-models","slug":"momentum-momentum-must-be-in-the-range-0-1","errorCode":null,"errorMessage":"Momentum {momentum} must be in the range [0,1]","messagePattern":"Momentum (.+?) must be in the range \\[0,1\\]","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"timm/optim/madgrad.py","lineNumber":65,"sourceCode":"        momentum (float):\n            Momentum value in  the range [0,1) (default: 0.9).\n        weight_decay (float):\n            Weight decay, i.e. a L2 penalty (default: 0).\n        eps (float):\n            Term added to the denominator outside of the root operation to improve numerical stability. (default: 1e-6).\n    \"\"\"\n\n    def __init__(\n            self,\n            params: _params_t,\n            lr: float = 1e-2,\n            momentum: float = 0.9,\n            weight_decay: float = 0,\n            eps: float = 1e-6,\n            decoupled_decay: bool = False,\n    ):\n        if momentum < 0 or momentum >= 1:\n            raise ValueError(f\"Momentum {momentum} must be in the range [0,1]\")\n        if lr <= 0:\n            raise ValueError(f\"Learning rate {lr} must be positive\")\n        if weight_decay < 0:\n            raise ValueError(f\"Weight decay {weight_decay} must be non-negative\")\n        if eps < 0:\n            raise ValueError(f\"Eps must be non-negative\")\n\n        defaults = dict(\n            lr=lr,\n            eps=eps,\n            momentum=momentum,\n            weight_decay=weight_decay,\n            decoupled_decay=decoupled_decay,\n        )\n        super().__init__(params, defaults)\n\n    @property\n    def supports_memory_efficient_fp16(self) -> bool:","sourceCodeStart":47,"sourceCodeEnd":83,"githubUrl":"https://github.com/huggingface/pytorch-image-models/blob/9a5261e31b3b5128526eb2658333b4c0a54464ae/timm/optim/madgrad.py#L47-L83","documentation":"MADGRAD optimizer validates that its momentum hyperparameter lies in [0, 1). Momentum of 1 or above, or a negative value, makes the momentum buffer diverge from the parameter trajectory, so the constructor rejects it immediately.","triggerScenarios":"Constructing madgrad.MADGRAD(params, momentum=1.0) or momentum=-0.1, or passing a momentum value sourced from a config/CLI without bounds checking. Note momentum=1 is rejected (range is [0,1) exclusive at the top).","commonSituations":"Copying momentum=0.999 or 1.0 settings from an Adam/SGD config where values close to 1 are common; sweeping momentum values without excluding the upper bound.","solutions":["Set momentum to a value in [0, 1), e.g. the default 0.9","If you copied momentum from another optimizer's config, re-tune it for MADGRAD (0.9 or 0.95 are typical)","Add bounds validation in your hyperparameter sweep/config loader"],"exampleFix":"// before\nopt = MADGRAD(model.parameters(), lr=1e-3, momentum=1.0)\n// after\nopt = MADGRAD(model.parameters(), lr=1e-3, momentum=0.9)","handlingStrategy":"validation","validationCode":"assert 0 <= cfg.momentum < 1, f'momentum {cfg.momentum} out of [0,1)'","typeGuard":"def is_valid_momentum(m: float) -> bool:\n    return isinstance(m, (int, float)) and 0 <= m < 1","tryCatchPattern":null,"preventionTips":["Clamp/validate momentum in config loaders before optimizer creation","Keep MADGRAD momentum at 0.9 unless tuned"],"tags":["optimizer","madgrad","momentum","hyperparameter-validation"],"backgroundTag":"hyperparameter-out-of-range","analyzedSha":"9a5261e31b3b5128526eb2658333b4c0a54464ae","analyzedAt":"2026-08-27T02:34:25.417Z","schemaVersion":2},"datasetVersion":"2026-08-27T03:17:27.898Z"}