{"record":{"id":"bef63e6be3ba305f","repo":"labmlai/annotated_deep_learning_paper_implementations","slug":"invalid-beta-parameter-at-index-1-betas-1","errorCode":null,"errorMessage":"Invalid beta parameter at index 1: {betas[1]}","messagePattern":"Invalid beta parameter at index 1: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"labml_nn/optimizers/__init__.py","lineNumber":94,"sourceCode":"        \"\"\"\n        ### Initialize\n\n        * `params` is the collection of parameters or set of parameter groups.\n        * `defaults` a dictionary of default hyper-parameters\n        * `lr` is the learning rate, $\\alpha$\n        * `betas` is the tuple $(\\beta_1, \\beta_2)$\n        * `eps` is $\\epsilon$\n        \"\"\"\n\n        # Check the hyper-parameters\n        if not 0.0 <= lr:\n            raise ValueError(f\"Invalid learning rate: {lr}\")\n        if not 0.0 <= eps:\n            raise ValueError(f\"Invalid epsilon value: {eps}\")\n        if not 0.0 <= betas[0] < 1.0:\n            raise ValueError(f\"Invalid beta parameter at index 0: {betas[0]}\")\n        if not 0.0 <= betas[1] < 1.0:\n            raise ValueError(f\"Invalid beta parameter at index 1: {betas[1]}\")\n\n        # Add the hyper-parameters to the defaults\n        defaults.update(dict(lr=lr, betas=betas, eps=eps))\n        # Initialize the PyTorch optimizer.\n        # This will create parameter groups with the default hyper-parameters\n        super().__init__(params, defaults)\n\n    def init_state(self, state: Dict[str, any], group: Dict[str, any], param: nn.Parameter):\n        \"\"\"\n        ### Initialize state for a given parameter tensor\n\n        This should be overridden with code to initialize `state` for parameters `param`.\n        `group` is the parameter group dictionary to which `param` belongs.\n        \"\"\"\n        pass\n\n    def step_param(self, state: Dict[str, any], group: Dict[str, any], grad: torch.Tensor, param: torch.Tensor):\n        \"\"\"","sourceCodeStart":76,"sourceCodeEnd":112,"githubUrl":"https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/33ab02281c2b928e6b32792909cc79cbdcfe1d6a/labml_nn/optimizers/__init__.py#L76-L112","documentation":"GenericAdaptiveOptimizer requires 0 <= betas[1] < 1. beta[1] (beta2) is the decay rate for the second-moment (squared-gradient moving average) estimate used in the adaptive denominator. beta2 >= 1 or negative values break convergence, so the constructor raises ValueError for them.","triggerScenarios":"Constructing the optimizer with betas[1] outside [0, 1), e.g. betas=(0.9, 1.0), betas=(0.9, 0.9999) is fine but betas=(0.9, -0.999) or (0.9, 1.1) raise.","commonSituations":"Typo turning 0.999 into 1.00 or 9.99; sweeps sampling beta2 over [0, 2]; configs migrated from optimizers that allowed beta2 = 1 (degenerate); YAML parsing 0.999 as a string.","solutions":["Use a valid beta2 in [0, 1), typically 0.999 or 0.98","Restrict the sweep/config range for beta2 to values strictly below 1.0","Add a pre-construction check on the betas tuple"],"exampleFix":"# before: beta2 = 1.0 -> raises\nopt = GenericAdaptiveOptimizer(params, lr=1e-3, betas=(0.9, 1.0))\n\n# after\nopt = GenericAdaptiveOptimizer(params, lr=1e-3, betas=(0.9, 0.999))","handlingStrategy":"validation","validationCode":"beta2 = cfg['betas'][1]\nif not 0.0 <= beta2 < 1.0:\n    raise ValueError(f'beta2 must be in [0, 1), got {beta2}')","typeGuard":"def valid_beta2(beta2: float) -> bool:\n    return isinstance(beta2, (int, float)) and 0.0 <= beta2 < 1.0","tryCatchPattern":"try:\n    opt = GenericAdaptiveOptimizer(params, lr=lr, betas=(beta1, beta2))\nexcept ValueError as e:\n    raise SystemExit(f'Bad optimizer config: {e}') from e","preventionTips":["Keep beta2 just below 1 (0.98–0.999); never exactly 1.0","Add a config-schema check that rejects beta2 >= 1.0 with a clear message"],"tags":["python","pytorch","optimizer","hyperparameter","validation"],"backgroundTag":"optimizer-hyperparameter-validation","analyzedSha":"33ab02281c2b928e6b32792909cc79cbdcfe1d6a","analyzedAt":"2026-08-25T10:30:27.743Z","schemaVersion":2},"datasetVersion":"2026-08-25T11:17:15.655Z"}