{"record":{"id":"34315c818f0fee5a","repo":"geekcomputers/Python","slug":"invalid-learning-rate-lr","errorCode":null,"errorMessage":"Invalid learning rate: {lr}","messagePattern":"Invalid learning rate: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"ML/src/python/neuralforge/optim/optimizers.py","lineNumber":8,"sourceCode":"import torch\nfrom torch.optim.optimizer import Optimizer\nimport math\n\nclass AdamW(Optimizer):\n    def __init__(self, params, lr=1e-3, betas=(0.9, 0.999), eps=1e-8, weight_decay=0.01, amsgrad=False):\n        if lr < 0.0:\n            raise ValueError(f\"Invalid learning rate: {lr}\")\n        if eps < 0.0:\n            raise ValueError(f\"Invalid epsilon value: {eps}\")\n        if not 0.0 <= betas[0] < 1.0:\n            raise ValueError(f\"Invalid beta parameter at index 0: {betas[0]}\")\n        if not 0.0 <= betas[1] < 1.0:\n            raise ValueError(f\"Invalid beta parameter at index 1: {betas[1]}\")\n        \n        defaults = dict(lr=lr, betas=betas, eps=eps, weight_decay=weight_decay, amsgrad=amsgrad)\n        super().__init__(params, defaults)\n    \n    def step(self, closure=None):\n        loss = None\n        if closure is not None:\n            loss = closure()\n        \n        for group in self.param_groups:\n            for p in group['params']:\n                if p.grad is None:","sourceCodeStart":1,"sourceCodeEnd":26,"githubUrl":"https://github.com/geekcomputers/Python/blob/40f4cd2652d75ef8e49d76e5c4d431d458712719/ML/src/python/neuralforge/optim/optimizers.py#L1-L26","documentation":"Raised by neuralforge's custom AdamW optimizer constructor when lr is negative. It mirrors torch.optim.AdamW's validation: learning rate must be non-negative. lr == 0 is accepted (only useful for schedules that will raise it later).","triggerScenarios":"Constructing AdamW(params, lr=-0.001); passing a value read from a config that defaulted to -1 as a placeholder; a learning-rate schedule or hyperparameter search proposing a negative value.","commonSituations":"Hyperparameter sweeps (optuna/raytune) sampling negative lr; config typos (negative sign); porting configs between libraries where lr semantics differ; deserialized checkpoint configs with sentinel values like -1.","solutions":["Inspect the lr value right before constructing the optimizer and log it","Clamp sweep/sample results: lr = max(lr, 0.0) or sample in log-space (e.g. 10**uniform(-5, -1))","Fix config typos or sentinel defaults like lr: -1","If lr legitimately starts at 0 for a scheduler, that is allowed; only negatives raise"],"exampleFix":"# before\nimport math\nlr = 10 ** trial.suggest_float('log_lr', -5, 1)  # can exceed safe range / sign bugs\nopt = AdamW(params, lr=-1e-3)  # ValueError\n\n# after\nlr = 10 ** trial.suggest_float('log_lr', -5, -3)\nopt = AdamW(params, lr=max(lr, 0.0))","handlingStrategy":"validation","validationCode":"assert lr >= 0, f'lr must be >= 0, got {lr}'\nopt = AdamW(params, lr=lr)","typeGuard":"def is_valid_lr(lr) -> bool:\n    return isinstance(lr, (int, float)) and lr >= 0","tryCatchPattern":"try:\n    opt = AdamW(params, lr=lr)\nexcept ValueError as e:\n    if 'Invalid learning rate' in str(e):\n        opt = AdamW(params, lr=abs(lr))  # or fall back to default 1e-3\n    else:\n        raise","preventionTips":["Sample hyperparameters in log-space to keep them positive","Validate config values at load time before training starts","Reject sentinel values like -1 in configs"],"tags":["optimizer","adamw","hyperparameter","validation","neuralforge"],"backgroundTag":"invalid-hyperparameter","analyzedSha":"40f4cd2652d75ef8e49d76e5c4d431d458712719","analyzedAt":"2026-08-27T11:12:20.313Z","schemaVersion":2},"datasetVersion":"2026-08-27T13:17:12.746Z"}