huggingface/pytorch-image-models · error · ValueError

Invalid learning rate: {lr}

Error message

Invalid learning rate: {lr}

What it means

Raised by ADOPT's constructor when the learning rate is negative (works for float lr via comparison, and tensor lr evaluates truthily per element context). The optimizer requires lr >= 0.

Source

Thrown at timm/optim/adopt.py:89

            weight_decay: float = 0.0,
            decoupled: bool = False,
            corrected_weight_decay: bool = False,
            *,
            caution: bool = False,
            foreach: Optional[bool] = False,
            maximize: bool = False,
            capturable: bool = False,
            differentiable: bool = False,
    ):
        if isinstance(lr, Tensor):
            if foreach and not capturable:
                raise ValueError(
                    "lr as a Tensor is not supported for capturable=False and foreach=True"
                )
            if lr.numel() != 1:
                raise ValueError("Tensor lr must be 1-element")
        if not 0.0 <= lr:
            raise ValueError(f"Invalid learning rate: {lr}")
        if not 0.0 <= eps:
            raise ValueError(f"Invalid epsilon value: {eps}")
        if not 0.0 <= betas[0] < 1.0:
            raise ValueError(f"Invalid beta parameter at index 0: {betas[0]}")
        if not 0.0 <= betas[1] < 1.0:
            raise ValueError(f"Invalid beta parameter at index 1: {betas[1]}")
        if not 0.0 <= weight_decay:
            raise ValueError(f"Invalid weight_decay value: {weight_decay}")

        defaults = dict(
            lr=lr,
            betas=betas,
            eps=eps,
            weight_decay=weight_decay,
            clip_exp=clip_exp,
            decoupled=decoupled,
            corrected_weight_decay=corrected_weight_decay,
            caution=caution,

View on GitHub (pinned to 9a5261e31b)

Solutions

  1. Use a non-negative lr, typically 1e-3 for ADOPT
  2. Check the config/sweep bounds and the value actually reaching the constructor

Example fix

# before
opt = timm.optim.Adopt(model.parameters(), lr=-1e-3)
# after
opt = timm.optim.Adopt(model.parameters(), lr=1e-3)
Defensive patterns

Strategy: validation

Validate before calling

lr_v = lr.item() if isinstance(lr, torch.Tensor) else lr
assert lr_v >= 0.0, f'lr must be >= 0, got {lr_v}'

Type guard

def valid_lr(lr) -> bool:
    v = lr.item() if isinstance(lr, torch.Tensor) else lr
    return v >= 0.0

Prevention

When it happens

Trigger: Constructing timm.optim.Adopt(params, lr=-1e-3), or a config/sweep supplying a negative lr.

Common situations: Sign typo in config, hyperparameter search ranges crossing zero, or misparsed CLI scientific notation.

Related errors

AI-assisted analysis of huggingface/pytorch-image-models@9a5261e31b (2026-08-27). Data as JSON: /api/errors/1b8e337670089a7f. Report an issue: GitHub.