geekcomputers/Python · error · ValueError

Invalid beta parameter at index 0: {betas[0]}

Error message

Invalid beta parameter at index 0: {betas[0]}

What it means

This ValueError is raised by the AdamW optimizer's constructor when the first momentum decay coefficient (betas[0]) falls outside the half-open range [0.0, 1.0). betas[0] controls the exponential decay rate for the first-moment (gradient mean) estimate. The check `0.0 <= betas[0] < 1.0` mirrors PyTorch's AdamW validation, rejecting values like 1.0 or negative numbers that would make the moving average degenerate or unstable.

Source

Thrown at ML/src/python/neuralforge/optim/optimizers.py:12

import torch
from torch.optim.optimizer import Optimizer
import math

class AdamW(Optimizer):
    def __init__(self, params, lr=1e-3, betas=(0.9, 0.999), eps=1e-8, weight_decay=0.01, amsgrad=False):
        if lr < 0.0:
            raise ValueError(f"Invalid learning rate: {lr}")
        if eps < 0.0:
            raise ValueError(f"Invalid epsilon value: {eps}")
        if not 0.0 <= betas[0] < 1.0:
            raise ValueError(f"Invalid beta parameter at index 0: {betas[0]}")
        if not 0.0 <= betas[1] < 1.0:
            raise ValueError(f"Invalid beta parameter at index 1: {betas[1]}")
        
        defaults = dict(lr=lr, betas=betas, eps=eps, weight_decay=weight_decay, amsgrad=amsgrad)
        super().__init__(params, defaults)
    
    def step(self, closure=None):
        loss = None
        if closure is not None:
            loss = closure()
        
        for group in self.param_groups:
            for p in group['params']:
                if p.grad is None:
                    continue
                
                grad = p.grad.data
                if grad.is_sparse:

View on GitHub (pinned to 40f4cd2652)

Solutions

1. Set betas[0] to the default 0.9 (e.g., betas=(0.9, 0.999))
2. Validate hyperparameter ranges before constructing the optimizer, clamping to [0.0, 1.0)
3. If loading config from file, ensure betas values are parsed as floats and sanity-checked
4. Check for swapped keyword arguments if the value looks like a learning rate

Example fix

// before
opt = AdamW(params, lr=1e-3, betas=(1.0, 0.999))

# after
opt = AdamW(params, lr=1e-3, betas=(0.9, 0.999))

betas = (0.9, 0.999)
assert all(0.0 <= b < 1.0 for b in betas), f'betas out of range: {betas}'
opt = AdamW(params, betas=betas)

def valid_betas(betas) -> bool:
    return (isinstance(betas, (tuple, list)) and len(betas) == 2
            and all(isinstance(b, (int, float)) and 0.0 <= b < 1.0 for b in betas))

try:
    opt = AdamW(params, betas=betas)
except ValueError as e:
    raise ValueError(f'Optimizer config invalid: {e}; using defaults') from e

When it happens

Trigger: Constructing the optimizer with betas=(1.0, 0.999), a negative beta like (-0.1, 0.999), or a beta >= 1.0 such as betas=(1.2, 0.999). Also happens when configuration is loaded from a file/env var and parsed as float without bounds checking, or when betas is accidentally reversed/malformed (e.g., passing lr into betas via keyword mix-ups).

Common situations: Hyperparameter sweeps that include 1.0 as an endpoint, YAML/JSON configs where betas is typed as a string, porting configs between frameworks with different beta conventions, or programmatic tuning that explores values outside [0,1).

Related errors

• Invalid beta parameter at index 1: {betas[1]}
• Invalid learning rate: {lr}
• Invalid epsilon value: {eps}
• AdamW does not support sparse gradients
• Please give a integer

AI-assisted analysis of geekcomputers/Python@40f4cd2652 (2026-08-27). Data as JSON: /api/errors/900301426e4d8c48. Report an issue: GitHub.