huggingface/pytorch-image-models · error · ValueError

Cannot compute fallback_lr_scale from adamw_lr when lr=0

Error message

Cannot compute fallback_lr_scale from adamw_lr when lr=0

What it means

When the deprecated adamw_lr is supplied, Muon derives fallback_lr_scale = adamw_lr / lr; with lr == 0 this division is undefined, so the constructor rejects the combination.

Source

Thrown at timm/optim/muon.py:747

        _validate_scalar("weight_decay", weight_decay)
        _validate_scalar("momentum", momentum, max_value=1.0)
        _validate_scalar("epsilon", eps)
        if conv_mode not in ["flatten", "batched"]:
            raise ValueError(f"Invalid conv_mode: {conv_mode}")
        if algo not in ["muon", "adamuon"]:
            raise ValueError(f"Invalid algo: {algo}. Must be 'muon' or 'adamuon'")

        if adamw_lr is not None:
            warnings.warn(
                "adamw_lr is deprecated, use fallback_lr_scale=adamw_lr/lr instead. "
                "adamw_lr will be removed in a future release.",
                FutureWarning,
                stacklevel=2,
            )
            if torch.is_tensor(lr):
                raise ValueError("adamw_lr is not supported with tensor lr; use fallback_lr_scale instead.")
            if lr == 0:
                raise ValueError("Cannot compute fallback_lr_scale from adamw_lr when lr=0")
            fallback_lr_scale = adamw_lr / lr

        defaults = dict(
            lr=lr,
            weight_decay=weight_decay,
            momentum=momentum,
            nesterov=nesterov,
            ns_steps=ns_steps,
            ns_coefficients=ns_coefficients,
            eps=eps,
            safety_factor=safety_factor,
            adjust_lr_fn=adjust_lr_fn,
            conv_mode=conv_mode,
            normalize_spatial=normalize_spatial,
            fallback_lr_scale=fallback_lr_scale,
            betas=betas,
            algo=algo,
            scale_eps=scale_eps,

View on GitHub (pinned to 9a5261e31b)

Solutions

1. Set fallback_lr_scale explicitly instead of adamw_lr
2. Use a nonzero lr (e.g. 1e-6) if a zero LR phase is needed, and manage warmup via a scheduler
3. Remove adamw_lr entirely if fallback scaling is not needed

Example fix

# before
opt = Muon(params, lr=0, adamw_lr=1e-4)
# after
opt = Muon(params, lr=1e-3, fallback_lr_scale=0.1)  # scale managed via scheduler

if cfg.get('adamw_lr') is not None:
    assert cfg['lr'] != 0 and not torch.is_tensor(cfg['lr'])

When it happens

Trigger: Passing adamw_lr together with lr=0 to the Muon constructor.

Common situations: Configs where lr is zeroed out (freeze phase, warmup start) while adamw_lr is still set from an earlier configuration.

Related errors

• adamw_lr is not supported with tensor lr; use fallback_lr_sc
• adamw_lr is deprecated, use fallback_lr_scale=adamw_lr/lr in
• Tensor must have at least 2 dimensions, got {tensor.ndim}
• Unknown mode: {mode}
• Invalid conv_mode: {conv_mode}

AI-assisted analysis of huggingface/pytorch-image-models@9a5261e31b (2026-08-27). Data as JSON: /api/errors/8cec042b9232ae4f. Report an issue: GitHub.