huggingface/pytorch-image-models · error · ValueError

adamw_lr is not supported with tensor lr; use fallback_lr_sc

Error message

adamw_lr is not supported with tensor lr; use fallback_lr_scale instead.

What it means

The deprecated adamw_lr argument cannot be combined with a tensor-valued lr, because the constructor would need to compute fallback_lr_scale = adamw_lr / lr via tensor division, which is unsupported; the code raises instead and tells you to use fallback_lr_scale.

Source

Thrown at timm/optim/muon.py:745

        """
        _validate_scalar("learning rate", lr)
        _validate_scalar("weight_decay", weight_decay)
        _validate_scalar("momentum", momentum, max_value=1.0)
        _validate_scalar("epsilon", eps)
        if conv_mode not in ["flatten", "batched"]:
            raise ValueError(f"Invalid conv_mode: {conv_mode}")
        if algo not in ["muon", "adamuon"]:
            raise ValueError(f"Invalid algo: {algo}. Must be 'muon' or 'adamuon'")

        if adamw_lr is not None:
            warnings.warn(
                "adamw_lr is deprecated, use fallback_lr_scale=adamw_lr/lr instead. "
                "adamw_lr will be removed in a future release.",
                FutureWarning,
                stacklevel=2,
            )
            if torch.is_tensor(lr):
                raise ValueError("adamw_lr is not supported with tensor lr; use fallback_lr_scale instead.")
            if lr == 0:
                raise ValueError("Cannot compute fallback_lr_scale from adamw_lr when lr=0")
            fallback_lr_scale = adamw_lr / lr

        defaults = dict(
            lr=lr,
            weight_decay=weight_decay,
            momentum=momentum,
            nesterov=nesterov,
            ns_steps=ns_steps,
            ns_coefficients=ns_coefficients,
            eps=eps,
            safety_factor=safety_factor,
            adjust_lr_fn=adjust_lr_fn,
            conv_mode=conv_mode,
            normalize_spatial=normalize_spatial,
            fallback_lr_scale=fallback_lr_scale,
            betas=betas,

View on GitHub (pinned to 9a5261e31b)

Solutions

1. Replace adamw_lr with fallback_lr_scale=adamw_lr/lr computed as a float before constructing the optimizer
2. Pass a scalar lr if you must keep adamw_lr (it will still emit a FutureWarning)
3. Upgrade to the non-deprecated API to silence the warning path entirely

Example fix

# before
opt = Muon(params, lr=lr_tensor, adamw_lr=1e-4)
# after
opt = Muon(params, lr=lr_tensor, fallback_lr_scale=1e-4 / base_lr)

if cfg.get('adamw_lr') is not None:
    assert not torch.is_tensor(cfg['lr']), 'use fallback_lr_scale with tensor lr'

def lr_is_tensor(x) -> bool:
    return torch.is_tensor(x)

When it happens

Trigger: Passing adamw_lr (deprecated) together with lr as a torch.Tensor (e.g. a schedule-free style tensor LR) to the Muon constructor.

Common situations: Migrating older timm Muon scripts that used adamw_lr while adopting tensor-LR training (schedule-free or per-step LR-as-tensor patterns).

Related errors

• Cannot compute fallback_lr_scale from adamw_lr when lr=0
• adamw_lr is deprecated, use fallback_lr_scale=adamw_lr/lr in
• lr as a Tensor is not supported for capturable=False and for
• Tensor lr must be 1-element
• Tensor must have at least 2 dimensions, got {tensor.ndim}

AI-assisted analysis of huggingface/pytorch-image-models@9a5261e31b (2026-08-27). Data as JSON: /api/errors/f49f71156112aa35. Report an issue: GitHub.