huggingface/pytorch-image-models · error · ValueError

Tensor lr must be 1-element

Error message

Tensor lr must be 1-element

What it means

ADOPT's constructor requires that a tensor learning rate contain exactly one element, since the update applies a single scalar lr across all parameters in the group.

Source

Thrown at timm/optim/adopt.py:87

            eps: float = 1e-6,
            clip_exp: Optional[float] = 0.333,
            weight_decay: float = 0.0,
            decoupled: bool = False,
            corrected_weight_decay: bool = False,
            *,
            caution: bool = False,
            foreach: Optional[bool] = False,
            maximize: bool = False,
            capturable: bool = False,
            differentiable: bool = False,
    ):
        if isinstance(lr, Tensor):
            if foreach and not capturable:
                raise ValueError(
                    "lr as a Tensor is not supported for capturable=False and foreach=True"
                )
            if lr.numel() != 1:
                raise ValueError("Tensor lr must be 1-element")
        if not 0.0 <= lr:
            raise ValueError(f"Invalid learning rate: {lr}")
        if not 0.0 <= eps:
            raise ValueError(f"Invalid epsilon value: {eps}")
        if not 0.0 <= betas[0] < 1.0:
            raise ValueError(f"Invalid beta parameter at index 0: {betas[0]}")
        if not 0.0 <= betas[1] < 1.0:
            raise ValueError(f"Invalid beta parameter at index 1: {betas[1]}")
        if not 0.0 <= weight_decay:
            raise ValueError(f"Invalid weight_decay value: {weight_decay}")

        defaults = dict(
            lr=lr,
            betas=betas,
            eps=eps,
            weight_decay=weight_decay,
            clip_exp=clip_exp,
            decoupled=decoupled,

View on GitHub (pinned to 9a5261e31b)

Solutions

1. Pass a 1-element tensor, e.g. torch.tensor(1e-3) or a (1,)-shaped schedule slice
2. Better: pass a float lr and update optimizer.param_groups[i]['lr'] each step with the scalar from your scheduler

Example fix

# before
opt = timm.optim.Adopt(model.parameters(), lr=torch.tensor([1e-3, 1e-4]))
# after
opt = timm.optim.Adopt(model.parameters(), lr=torch.tensor(1e-3))

if isinstance(lr, torch.Tensor):
    assert lr.numel() == 1, 'tensor lr must be scalar'

def is_scalar_lr(lr) -> bool:
    return not isinstance(lr, torch.Tensor) or lr.numel() == 1

When it happens

Trigger: Constructing timm.optim.Adopt(params, lr=torch.tensor([1e-3, 1e-4])) — any tensor lr with lr.numel() != 1.

Common situations: Passing a per-layer or per-step lr schedule as a multi-element tensor instead of slicing out one scalar per step, or accidentally passing a batch-shaped tensor as lr.

Related errors

• lr as a Tensor is not supported for capturable=False and for
• Invalid learning rate: {lr}
• Invalid epsilon value: {eps}
• Invalid beta parameter at index 0: {betas[0]}
• Invalid beta parameter at index 1: {betas[1]}

AI-assisted analysis of huggingface/pytorch-image-models@9a5261e31b (2026-08-27). Data as JSON: /api/errors/d4d8009d8f8cd882. Report an issue: GitHub.