geekcomputers/Python · error · ValueError

Invalid epsilon value: {eps}

Error message

Invalid epsilon value: {eps}

What it means

Raised by neuralforge's custom AdamW constructor when eps is negative. Like the lr check, it validates the denominator-stability constant up front; eps of exactly 0 is allowed, only negative values raise ValueError.

Source

Thrown at ML/src/python/neuralforge/optim/optimizers.py:10

import torch
from torch.optim.optimizer import Optimizer
import math

class AdamW(Optimizer):
    def __init__(self, params, lr=1e-3, betas=(0.9, 0.999), eps=1e-8, weight_decay=0.01, amsgrad=False):
        if lr < 0.0:
            raise ValueError(f"Invalid learning rate: {lr}")
        if eps < 0.0:
            raise ValueError(f"Invalid epsilon value: {eps}")
        if not 0.0 <= betas[0] < 1.0:
            raise ValueError(f"Invalid beta parameter at index 0: {betas[0]}")
        if not 0.0 <= betas[1] < 1.0:
            raise ValueError(f"Invalid beta parameter at index 1: {betas[1]}")
        
        defaults = dict(lr=lr, betas=betas, eps=eps, weight_decay=weight_decay, amsgrad=amsgrad)
        super().__init__(params, defaults)
    
    def step(self, closure=None):
        loss = None
        if closure is not None:
            loss = closure()
        
        for group in self.param_groups:
            for p in group['params']:
                if p.grad is None:
                    continue
                

View on GitHub (pinned to 40f4cd2652)

Solutions

1. Log the eps value passed to the constructor
2. Sample eps in log-space or a strictly positive range: eps = abs(eps) or 10**uniform(-9, -7)
3. Fix the config typo
4. If eps=0 is intended (rare, risks division by zero), it passes validation but consider a small positive floor

Example fix

# before
opt = AdamW(params, eps=-1e-8)  # ValueError

# after
eps = trial.suggest_float('log_eps', -9, -7)
opt = AdamW(params, eps=10 ** eps)

assert eps >= 0, f'eps must be >= 0, got {eps}'
opt = AdamW(params, eps=eps)

def is_valid_eps(eps) -> bool:
    return isinstance(eps, (int, float)) and eps >= 0

try:
    opt = AdamW(params, eps=eps)
except ValueError as e:
    if 'Invalid epsilon' in str(e):
        opt = AdamW(params, eps=abs(eps) or 1e-8)
    else:
        raise

When it happens

Trigger: Constructing AdamW(params, eps=-1e-8); loading eps from a config with a stray minus sign; hyperparameter search sampling eps from a symmetric range around zero.

Common situations: Typo'd configs; sweeps that sample eps uniformly in [-1e-8, 1e-6]; copying settings from another library with different sign conventions; checkpoint config deserialization mangling values.

Related errors

• Invalid learning rate: {lr}
• Invalid beta parameter at index 0: {betas[0]}
• Invalid beta parameter at index 1: {betas[1]}
• AdamW does not support sparse gradients
• Please give a integer

AI-assisted analysis of geekcomputers/Python@40f4cd2652 (2026-08-27). Data as JSON: /api/errors/3847f2e8f7043bc0. Report an issue: GitHub.