{"record":{"id":"8321936b732da7df","repo":"labmlai/annotated_deep_learning_paper_implementations","slug":"invalid-epsilon-value-eps","errorCode":null,"errorMessage":"Invalid epsilon value: {eps}","messagePattern":"Invalid epsilon value: (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"labml_nn/optimizers/__init__.py","lineNumber":90,"sourceCode":"    ## Base class for *Adam* and extensions\n    \"\"\"\n\n    def __init__(self, params, defaults: Dict[str, Any], lr: float, betas: Tuple[float, float], eps: float):\n        \"\"\"\n        ### Initialize\n\n        * `params` is the collection of parameters or set of parameter groups.\n        * `defaults` a dictionary of default hyper-parameters\n        * `lr` is the learning rate, $\\alpha$\n        * `betas` is the tuple $(\\beta_1, \\beta_2)$\n        * `eps` is $\\epsilon$\n        \"\"\"\n\n        # Check the hyper-parameters\n        if not 0.0 <= lr:\n            raise ValueError(f\"Invalid learning rate: {lr}\")\n        if not 0.0 <= eps:\n            raise ValueError(f\"Invalid epsilon value: {eps}\")\n        if not 0.0 <= betas[0] < 1.0:\n            raise ValueError(f\"Invalid beta parameter at index 0: {betas[0]}\")\n        if not 0.0 <= betas[1] < 1.0:\n            raise ValueError(f\"Invalid beta parameter at index 1: {betas[1]}\")\n\n        # Add the hyper-parameters to the defaults\n        defaults.update(dict(lr=lr, betas=betas, eps=eps))\n        # Initialize the PyTorch optimizer.\n        # This will create parameter groups with the default hyper-parameters\n        super().__init__(params, defaults)\n\n    def init_state(self, state: Dict[str, any], group: Dict[str, any], param: nn.Parameter):\n        \"\"\"\n        ### Initialize state for a given parameter tensor\n\n        This should be overridden with code to initialize `state` for parameters `param`.\n        `group` is the parameter group dictionary to which `param` belongs.\n        \"\"\"","sourceCodeStart":72,"sourceCodeEnd":108,"githubUrl":"https://github.com/labmlai/annotated_deep_learning_paper_implementations/blob/33ab02281c2b928e6b32792909cc79cbdcfe1d6a/labml_nn/optimizers/__init__.py#L72-L108","documentation":"GenericAdaptiveOptimizer's constructor requires eps >= 0 because epsilon is added to denominators for numerical stability. A negative eps would corrupt the adaptive scaling, so the constructor raises ValueError for any negative epsilon.","triggerScenarios":"Constructing the optimizer with eps < 0, e.g. GenericAdaptiveOptimizer(params, eps=-1e-8), or any labml-nn Adam variant with a negative epsilon from config/sweep.","commonSituations":"Config typo on the exponent (1e-8 mistyped as -1e-8); hyperparameter search with a symmetric range around zero; copy-paste from a paper table where eps was listed with a dash; positional-argument mixups.","solutions":["Check the eps value in your config; use a positive epsilon such as 1e-8","Constrain sweep ranges for eps to positive values","Add a startup assertion on all hyper-parameters before building the optimizer"],"exampleFix":"# before\nopt = GenericAdaptiveOptimizer(model.parameters(), lr=1e-3, eps=-1e-8)\n\n# after\nopt = GenericAdaptiveOptimizer(model.parameters(), lr=1e-3, eps=1e-8)","handlingStrategy":"validation","validationCode":"eps = float(cfg['eps'])\nif not 0.0 <= eps:\n    raise ValueError(f'config eps must be >= 0, got {eps}')","typeGuard":"def valid_eps(eps: float) -> bool:\n    return isinstance(eps, (int, float)) and 0.0 <= eps < float('inf')","tryCatchPattern":"try:\n    opt = GenericAdaptiveOptimizer(params, lr=lr, eps=eps)\nexcept ValueError as e:\n    raise SystemExit(f'Bad optimizer config: {e}') from e","preventionTips":["Default eps to 1e-8 unless a paper specifies otherwise","Reject negative values in config schema validation before training starts"],"tags":["python","pytorch","optimizer","hyperparameter","validation"],"backgroundTag":"optimizer-hyperparameter-validation","analyzedSha":"33ab02281c2b928e6b32792909cc79cbdcfe1d6a","analyzedAt":"2026-08-25T10:30:27.743Z","schemaVersion":2},"datasetVersion":"2026-08-25T11:17:15.655Z"}