{"record":{"id":"b90fc61fe661bc45","repo":"lllyasviel/ControlNet","slug":"invalid-ema-decay-value","errorCode":null,"errorMessage":"Invalid ema_decay value: {}","messagePattern":"Invalid ema_decay value: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"ldm/util.py","lineNumber":107,"sourceCode":"\nclass AdamWwithEMAandWings(optim.Optimizer):\n    # credit to https://gist.github.com/crowsonkb/65f7265353f403714fce3b2595e0b298\n    def __init__(self, params, lr=1.e-3, betas=(0.9, 0.999), eps=1.e-8,  # TODO: check hyperparameters before using\n                 weight_decay=1.e-2, amsgrad=False, ema_decay=0.9999,   # ema decay to match previous code\n                 ema_power=1., param_names=()):\n        \"\"\"AdamW that saves EMA versions of the parameters.\"\"\"\n        if not 0.0 <= lr:\n            raise ValueError(\"Invalid learning rate: {}\".format(lr))\n        if not 0.0 <= eps:\n            raise ValueError(\"Invalid epsilon value: {}\".format(eps))\n        if not 0.0 <= betas[0] < 1.0:\n            raise ValueError(\"Invalid beta parameter at index 0: {}\".format(betas[0]))\n        if not 0.0 <= betas[1] < 1.0:\n            raise ValueError(\"Invalid beta parameter at index 1: {}\".format(betas[1]))\n        if not 0.0 <= weight_decay:\n            raise ValueError(\"Invalid weight_decay value: {}\".format(weight_decay))\n        if not 0.0 <= ema_decay <= 1.0:\n            raise ValueError(\"Invalid ema_decay value: {}\".format(ema_decay))\n        defaults = dict(lr=lr, betas=betas, eps=eps,\n                        weight_decay=weight_decay, amsgrad=amsgrad, ema_decay=ema_decay,\n                        ema_power=ema_power, param_names=param_names)\n        super().__init__(params, defaults)\n\n    def __setstate__(self, state):\n        super().__setstate__(state)\n        for group in self.param_groups:\n            group.setdefault('amsgrad', False)\n\n    @torch.no_grad()\n    def step(self, closure=None):\n        \"\"\"Performs a single optimization step.\n        Args:\n            closure (callable, optional): A closure that reevaluates the model\n                and returns the loss.\n        \"\"\"\n        loss = None","sourceCodeStart":89,"sourceCodeEnd":125,"githubUrl":"https://github.com/lllyasviel/ControlNet/blob/ed85cd1e25a5ed592f7d8178495b4483de0331bf/ldm/util.py#L89-L125","documentation":"Raised by the EMA-tracking AdamW optimizer when ema_decay is outside [0.0, 1.0]. This custom parameter tracks an exponential moving average of the weights during training (used at inference for smoothed weights), so its decay factor must be a valid probability in the closed interval [0, 1].","triggerScenarios":"Constructing AdamW(params, ema_decay=1.2) or ema_decay=-0.5, or computing ema_decay dynamically (e.g. d * scale in EMA-adjustment logic) so that it leaves the valid range.","commonSituations":"Custom training loops that ramp EMA decay over time and overshoot 1.0, configs mixing up ema_decay with ema_power, or ported hyperparameters from torch_ema where valid ranges differ.","solutions":["Set ema_decay to a value in [0, 1]; the constructor default is 0.9999","If ramping decay per step, clamp: ema_decay = min(max(decay, 0.0), 1.0)","Double-check you didn't pass ema_power (typically 1.0 or 3/4) into the ema_decay slot"],"exampleFix":"# before\nopt = AdamW(params, ema_decay=min(1.0 + k * 0.01, ...))  # can exceed 1.0\n# after\nopt = AdamW(params, ema_decay=min(max(0.9999 + k * 1e-5, 0.0), 1.0))","handlingStrategy":"validation","validationCode":"assert 0.0 <= ema_decay <= 1.0, f'bad ema_decay: {ema_decay}'","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Clamp any schedule-computed decay: min(max(d, 0.0), 1.0)","Don't confuse ema_decay with ema_power","Keep EMA decay near 0.9999 unless deliberately tuning"],"tags":["optimizer","adamw","ema","diffusion-training","validation"],"backgroundTag":"invalid-optimizer-hyperparameter","analyzedSha":"ed85cd1e25a5ed592f7d8178495b4483de0331bf","analyzedAt":"2026-08-27T12:58:54.167Z","schemaVersion":2},"datasetVersion":"2026-08-27T13:17:12.746Z"}