{"record":{"id":"ce3add3387557a61","repo":"labmlai/annotated_deep_learning_paper_implementations","slug":"invalid-beta-parameter-at-index-0-betas-0","errorCode":null,"errorMessage":"Invalid beta parameter at index 0: {betas[0]}","messagePattern":"Invalid beta parameter at index 0: (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"labml_nn/optimizers/__init__.py","lineNumber":92,"sourceCode":"\n    def __init__(self, params, defaults: Dict[str, Any], lr: float, betas: Tuple[float, float], eps: float):\n        \"\"\"\n        ### Initialize\n\n        * `params` is the collection of parameters or set of parameter groups.\n        * `defaults` a dictionary of default hyper-parameters\n        * `lr` is the learning rate, $\\alpha$\n        * `betas` is the tuple $(\\beta_1, \\beta_2)$\n        * `eps` is $\\epsilon$\n        \"\"\"\n\n        # Check the hyper-parameters\n        if not 0.0 <= lr:\n            raise ValueError(f\"Invalid learning rate: {lr}\")\n        if not 0.0 <= eps:\n            raise ValueError(f\"Invalid epsilon value: {eps}\")\n        if not 0.0 <= betas[0] < 1.0:\n            raise ValueError(f\"Invalid beta parameter at index 0: {betas[0]}\")\n        if not 0.0 <= betas[1] < 1.0:\n            raise ValueError(f\"Invalid beta parameter at index 1: {betas[1]}\")\n\n        # Add the hyper-parameters to the defaults\n        defaults.update(dict(lr=lr, betas=betas, eps=eps))\n        # Initialize the PyTorch optimizer.\n        # This will create parameter groups with the default hyper-parameters\n        super().__init__(params, defaults)\n\n    def init_state(self, state: Dict[str, any], group: Dict[str, any], param: nn.Parameter):\n        \"\"\"\n        ### Initialize state for a given parameter tensor\n\n        This should be overridden with code to initialize `state` for parameters `param`.\n        `group` is the parameter group dictionary to which `param` belongs.\n        \"\"\"\n        pass\n","sourceCodeStart":74,"sourceCodeEnd":110,"githubUrl":"https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/33ab02281c2b928e6b32792909cc79cbdcfe1d6a/labml_nn/optimizers/__init__.py#L74-L110","documentation":"GenericAdaptiveOptimizer requires each momentum coefficient beta to satisfy 0 <= beta < 1. beta[0] (beta1) is the exponential decay rate for the first-moment (gradient moving average) estimate; values >= 1.0 or negative values make the moving average mathematically invalid (non-decaying or diverging), so the constructor rejects them.","triggerScenarios":"Constructing the optimizer with betas[0] outside [0, 1), e.g. betas=(1.0, 0.999), betas=(-0.9, 0.999), or betas=(1.1, 0.98).","commonSituations":"Copy-paste where 0.9 becomes 1.0 or 9.0; hyperparameter sweep sampling outside the valid range; misreading a paper's beta1; accidentally passing a tuple with swapped or extra elements.","solutions":["Use a valid beta1 in [0, 1), typically 0.9","Fix the config/sweep range for beta1 to [0.0, 1.0) exclusive","Validate betas before construction: assert all(0.0 <= b < 1.0 for b in betas)"],"exampleFix":"# before: beta1 = 1.0 -> raises\nopt = GenericAdaptiveOptimizer(params, lr=1e-3, betas=(1.0, 0.999))\n\n# after\nopt = GenericAdaptiveOptimizer(params, lr=1e-3, betas=(0.9, 0.999))","handlingStrategy":"validation","validationCode":"beta1, beta2 = cfg['betas']\nif not (0.0 <= beta1 < 1.0 and 0.0 <= beta2 < 1.0):\n    raise ValueError(f'betas must be in [0, 1), got {(beta1, beta2)}')","typeGuard":"def valid_betas(betas: tuple) -> bool:\n    return (len(betas) == 2\n            and all(isinstance(b, (int, float)) for b in betas)\n            and all(0.0 <= b < 1.0 for b in betas))","tryCatchPattern":"try:\n    opt = GenericAdaptiveOptimizer(params, lr=lr, betas=betas)\nexcept ValueError as e:\n    raise SystemExit(f'Bad optimizer config: {e}') from e","preventionTips":["Use the standard betas=(0.9, 0.999) unless tuning deliberately","Bound sweep ranges for betas strictly inside [0, 1)","Validate the betas tuple shape (exactly two numbers) at config load"],"tags":["python","pytorch","optimizer","hyperparameter","validation"],"backgroundTag":"optimizer-hyperparameter-validation","analyzedSha":"33ab02281c2b928e6b32792909cc79cbdcfe1d6a","analyzedAt":"2026-08-25T10:30:27.743Z","schemaVersion":2},"datasetVersion":"2026-08-25T11:17:15.655Z"}