{"record":{"id":"1c61ce31c4739510","repo":"lllyasviel/ControlNet","slug":"invalid-learning-rate","errorCode":null,"errorMessage":"Invalid learning rate: {}","messagePattern":"Invalid learning rate: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"ldm/util.py","lineNumber":97,"sourceCode":"    return get_obj_from_str(config[\"target\"])(**config.get(\"params\", dict()))\n\n\ndef get_obj_from_str(string, reload=False):\n    module, cls = string.rsplit(\".\", 1)\n    if reload:\n        module_imp = importlib.import_module(module)\n        importlib.reload(module_imp)\n    return getattr(importlib.import_module(module, package=None), cls)\n\n\nclass AdamWwithEMAandWings(optim.Optimizer):\n    # credit to https://gist.github.com/crowsonkb/65f7265353f403714fce3b2595e0b298\n    def __init__(self, params, lr=1.e-3, betas=(0.9, 0.999), eps=1.e-8,  # TODO: check hyperparameters before using\n                 weight_decay=1.e-2, amsgrad=False, ema_decay=0.9999,   # ema decay to match previous code\n                 ema_power=1., param_names=()):\n        \"\"\"AdamW that saves EMA versions of the parameters.\"\"\"\n        if not 0.0 <= lr:\n            raise ValueError(\"Invalid learning rate: {}\".format(lr))\n        if not 0.0 <= eps:\n            raise ValueError(\"Invalid epsilon value: {}\".format(eps))\n        if not 0.0 <= betas[0] < 1.0:\n            raise ValueError(\"Invalid beta parameter at index 0: {}\".format(betas[0]))\n        if not 0.0 <= betas[1] < 1.0:\n            raise ValueError(\"Invalid beta parameter at index 1: {}\".format(betas[1]))\n        if not 0.0 <= weight_decay:\n            raise ValueError(\"Invalid weight_decay value: {}\".format(weight_decay))\n        if not 0.0 <= ema_decay <= 1.0:\n            raise ValueError(\"Invalid ema_decay value: {}\".format(ema_decay))\n        defaults = dict(lr=lr, betas=betas, eps=eps,\n                        weight_decay=weight_decay, amsgrad=amsgrad, ema_decay=ema_decay,\n                        ema_power=ema_power, param_names=param_names)\n        super().__init__(params, defaults)\n\n    def __setstate__(self, state):\n        super().__setstate__(state)\n        for group in self.param_groups:","sourceCodeStart":79,"sourceCodeEnd":115,"githubUrl":"https://github.com/lllyasviel/ControlNet/blob/ed85cd1e25a5ed592f7d8178495b4483de0331bf/ldm/util.py#L79-L115","documentation":"Raised by the EMA-tracking AdamW optimizer in ldm/util.py when the learning rate passed to __init__ is negative (lr < 0.0). This mirrors torch.optim.AdamW's built-in argument validation; lr == 0 is allowed but any negative value aborts construction.","triggerScenarios":"Constructing this AdamW with a negative lr, e.g. AdamW(params, lr=-1e-3), or reading lr from a config/YAML where the value is negative (mistyped sign, subtraction bug, or a schedule that computed a negative value).","commonSituations":"Diffusion training scripts that compute lr from a schedule or multiply by a factor that underflows past zero, typo'd YAML values, or CLI arg parsing that passes a negated number.","solutions":["Check the lr value at the call site and fix the sign (e.g. 1e-4 not -1e-4)","If lr comes from a config, validate/clamp it: max(lr, 0.0) only if a floor is intended; otherwise fix the source of the value","If lr is computed by a scheduler lambda, guard the lambda to return max(new_lr, 0.0)"],"exampleFix":"# before\nopt = AdamW(params, lr=-1e-4)\n# after\nopt = AdamW(params, lr=1e-4)","handlingStrategy":"validation","validationCode":"assert 0.0 <= lr, f'bad lr: {lr}'","typeGuard":null,"tryCatchPattern":"try:\n    opt = AdamW(params, lr=lr)\nexcept ValueError as e:\n    if 'learning rate' in str(e):\n        raise ValueError(f'Fix learning rate config: lr={lr}')\n    raise","preventionTips":["Validate optimizer hyperparams in one place before constructing","Use bounded sweep ranges for lr (e.g. [1e-6, 1e-2])","Log the effective optimizer kwargs at training start"],"tags":["optimizer","adamw","learning-rate","diffusion-training","validation"],"backgroundTag":"invalid-optimizer-hyperparameter","analyzedSha":"ed85cd1e25a5ed592f7d8178495b4483de0331bf","analyzedAt":"2026-08-27T12:58:54.167Z","schemaVersion":2},"datasetVersion":"2026-08-27T13:17:12.746Z"}