{"record":{"id":"5e5cffce44173799","repo":"huggingface/pytorch-image-models","slug":"invalid-learning-rate-5e5cff","errorCode":null,"errorMessage":"Invalid learning rate: {}","messagePattern":"Invalid learning rate: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"timm/optim/nadam.py","lineNumber":42,"sourceCode":"\n    __ http://cs229.stanford.edu/proj2015/054_report.pdf\n    __ http://www.cs.toronto.edu/~fritz/absps/momentum.pdf\n\n        Originally taken from: https://github.com/pytorch/pytorch/pull/1408\n        NOTE: Has potential issues but does work well on some problems.\n    \"\"\"\n\n    def __init__(\n            self,\n            params,\n            lr=2e-3,\n            betas=(0.9, 0.999),\n            eps=1e-8,\n            weight_decay=0,\n            schedule_decay=4e-3,\n    ):\n        if not 0.0 <= lr:\n            raise ValueError(\"Invalid learning rate: {}\".format(lr))\n        defaults = dict(\n            lr=lr,\n            betas=betas,\n            eps=eps,\n            weight_decay=weight_decay,\n            schedule_decay=schedule_decay,\n        )\n        super(NAdamLegacy, self).__init__(params, defaults)\n\n    @torch.no_grad()\n    def step(self, closure=None):\n        \"\"\"Performs a single optimization step.\n\n        Arguments:\n            closure (callable, optional): A closure that reevaluates the model\n                and returns the loss.\n        \"\"\"\n        loss = None","sourceCodeStart":24,"sourceCodeEnd":60,"githubUrl":"https://github.com/huggingface/pytorch-image-models/blob/9a5261e31b3b5128526eb2658333b4c0a54464ae/timm/optim/nadam.py#L24-L60","documentation":"NAdam optimizer constructor validation: learning rate must satisfy 0.0 <= lr. A negative lr is rejected immediately at timm.optim.NAdam instantiation.","triggerScenarios":"Calling timm.optim.NAdam(params, lr=-0.01) or any negative learning rate; also NaN lr from a miscomputed config.","commonSituations":"Typo'd hyperparameter in a config file (lr: -1e-3), sign errors when negating lr for LR-sweep scripts, or lr loaded as negative from a CLI arg parsed incorrectly.","solutions":["Fix lr to a positive value such as 1e-3","Check config/CLI parsing for stray minus signs or unit mistakes","If lr is computed (e.g. scaled), clamp or assert lr >= 0 before constructing"],"exampleFix":"# before\nopt = NAdam(model.parameters(), lr=-1e-3)\n\n# after\nopt = NAdam(model.parameters(), lr=1e-3)","handlingStrategy":"validation","validationCode":"if not 0.0 <= lr:\n    raise ValueError(f'bad lr from config: {lr}')","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Validate hyperparameters after loading configs","Use typed config schemas that enforce lr >= 0"],"tags":["timm","nadam","learning-rate","valueerror"],"backgroundTag":"invalid-optimizer-hyperparameter","analyzedSha":"9a5261e31b3b5128526eb2658333b4c0a54464ae","analyzedAt":"2026-08-27T02:34:25.417Z","schemaVersion":2},"datasetVersion":"2026-08-27T03:17:27.898Z"}