geekcomputers/Python · error · ValueError

Invalid beta parameter at index 1: {betas[1]}

Error message

Invalid beta parameter at index 1: {betas[1]}

What it means

This ValueError is raised by the AdamW optimizer's constructor when the second momentum decay coefficient (betas[1]) falls outside [0.0, 1.0). betas[1] controls the exponential decay rate for the second-moment (squared gradient) estimate used in the bias-corrected denominator. Values of 1.0 or greater would prevent the bias correction from ever stabilizing, and negative values are mathematically meaningless for a decay average.

Source

Thrown at ML/src/python/neuralforge/optim/optimizers.py:14

import torch
from torch.optim.optimizer import Optimizer
import math

class AdamW(Optimizer):
    def __init__(self, params, lr=1e-3, betas=(0.9, 0.999), eps=1e-8, weight_decay=0.01, amsgrad=False):
        if lr < 0.0:
            raise ValueError(f"Invalid learning rate: {lr}")
        if eps < 0.0:
            raise ValueError(f"Invalid epsilon value: {eps}")
        if not 0.0 <= betas[0] < 1.0:
            raise ValueError(f"Invalid beta parameter at index 0: {betas[0]}")
        if not 0.0 <= betas[1] < 1.0:
            raise ValueError(f"Invalid beta parameter at index 1: {betas[1]}")
        
        defaults = dict(lr=lr, betas=betas, eps=eps, weight_decay=weight_decay, amsgrad=amsgrad)
        super().__init__(params, defaults)
    
    def step(self, closure=None):
        loss = None
        if closure is not None:
            loss = closure()
        
        for group in self.param_groups:
            for p in group['params']:
                if p.grad is None:
                    continue
                
                grad = p.grad.data
                if grad.is_sparse:
                    raise RuntimeError('AdamW does not support sparse gradients')
                

View on GitHub (pinned to 40f4cd2652)

Solutions

1. Use the default betas=(0.9, 0.999) or clamp beta2 strictly below 1.0
2. If annealing beta2, cap the schedule at e.g. 0.9999 instead of 1.0
3. Validate loaded config values before passing them to the optimizer constructor

Example fix

# before
opt = AdamW(params, betas=(0.9, beta2))  # beta2 may be 1.0

# after
beta2 = min(beta2, 0.9999)
opt = AdamW(params, betas=(0.9, beta2))

beta2 = min(float(beta2), 0.9999)
assert 0.0 <= beta2 < 1.0
opt = AdamW(params, betas=(0.9, beta2))

def valid_beta2(b) -> bool:
    return isinstance(b, (int, float)) and 0.0 <= float(b) < 1.0

try:
    opt = AdamW(params, betas=(beta1, beta2))
except ValueError:
    beta2 = min(beta2, 0.9999)
    opt = AdamW(params, betas=(beta1, beta2))

When it happens

Trigger: Constructing the optimizer with betas=(0.9, 1.0) or any second element >= 1.0 or < 0.0, e.g. betas=(0.9, -0.5). Common in schedules where beta2 is annealed toward 1.0 and the endpoint is hit exactly, or when a config value is mistyped.

Common situations: Beta2 annealing schedules (e.g., increasing beta2 during training) that reach 1.0, hyperparameter search boundaries including 1.0, config files with typos, or copy-paste from tutorials using nonstandard betas.

Related errors

• Invalid beta parameter at index 0: {betas[0]}
• Invalid learning rate: {lr}
• Invalid epsilon value: {eps}
• AdamW does not support sparse gradients
• Please give a integer

AI-assisted analysis of geekcomputers/Python@40f4cd2652 (2026-08-27). Data as JSON: /api/errors/1660b3e434d4962a. Report an issue: GitHub.