{"record":{"id":"1fd4119ef54f2b49","repo":"huggingface/pytorch-image-models","slug":"learning-rate-lr-must-be-positive","errorCode":null,"errorMessage":"Learning rate {lr} must be positive","messagePattern":"Learning rate (.+?) must be positive","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"timm/optim/madgrad.py","lineNumber":67,"sourceCode":"        weight_decay (float):\n            Weight decay, i.e. a L2 penalty (default: 0).\n        eps (float):\n            Term added to the denominator outside of the root operation to improve numerical stability. (default: 1e-6).\n    \"\"\"\n\n    def __init__(\n            self,\n            params: _params_t,\n            lr: float = 1e-2,\n            momentum: float = 0.9,\n            weight_decay: float = 0,\n            eps: float = 1e-6,\n            decoupled_decay: bool = False,\n    ):\n        if momentum < 0 or momentum >= 1:\n            raise ValueError(f\"Momentum {momentum} must be in the range [0,1]\")\n        if lr <= 0:\n            raise ValueError(f\"Learning rate {lr} must be positive\")\n        if weight_decay < 0:\n            raise ValueError(f\"Weight decay {weight_decay} must be non-negative\")\n        if eps < 0:\n            raise ValueError(f\"Eps must be non-negative\")\n\n        defaults = dict(\n            lr=lr,\n            eps=eps,\n            momentum=momentum,\n            weight_decay=weight_decay,\n            decoupled_decay=decoupled_decay,\n        )\n        super().__init__(params, defaults)\n\n    @property\n    def supports_memory_efficient_fp16(self) -> bool:\n        return False\n","sourceCodeStart":49,"sourceCodeEnd":85,"githubUrl":"https://github.com/huggingface/pytorch-image-models/blob/9a5261e31b3b5128526eb2658333b4c0a54464ae/timm/optim/madgrad.py#L49-L85","documentation":"MADGRAD requires a strictly positive learning rate because its update rule divides by quantities derived from lr; a zero or negative lr is meaningless and is rejected at construction time.","triggerScenarios":"Calling MADGRAD(params, lr=0) or lr=-0.1, e.g. when lr comes from a sweep that includes 0, or when a config key is missing and defaults to 0.","commonSituations":"Hyperparameter searches that probe lr=0; misparsed CLI args; a scheduler/JSON config typo producing 0 or a negative float.","solutions":["Pass a positive lr such as 1e-3 (MADGRAD's typical default)","Fix the config/CLI plumbing that produced 0 or a negative value","Exclude lr<=0 from sweep grids"],"exampleFix":"# before\nopt = MADGRAD(model.parameters(), lr=0)\n# after\nopt = MADGRAD(model.parameters(), lr=1e-3)","handlingStrategy":"validation","validationCode":"assert cfg.lr > 0, 'lr must be positive for MADGRAD'","typeGuard":"def is_valid_lr(lr: float) -> bool:\n    return isinstance(lr, (int, float)) and lr > 0","tryCatchPattern":null,"preventionTips":["Validate lr at config parse time","Exclude lr<=0 from sweep grids"],"tags":["optimizer","madgrad","learning-rate","hyperparameter-validation"],"backgroundTag":"hyperparameter-out-of-range","analyzedSha":"9a5261e31b3b5128526eb2658333b4c0a54464ae","analyzedAt":"2026-08-27T02:34:25.417Z","schemaVersion":2},"datasetVersion":"2026-08-27T03:17:27.898Z"}