huggingface/pytorch-image-models · warning · FutureWarning

adamw_lr is deprecated, use fallback_lr_scale=adamw_lr/lr in

Error message

adamw_lr is deprecated, use fallback_lr_scale=adamw_lr/lr instead. adamw_lr will be removed in a future release.

What it means

Muon's __init__ warns that the adamw_lr argument (separate LR for params handled by the AdamW fallback path) is deprecated. It will be removed; express it as a scale of the main lr via fallback_lr_scale = adamw_lr / lr.

Source

Thrown at timm/optim/muon.py:738

            # Manual control over parameter groups
            optimizer = Muon([
                {'params': weight_matrices, 'lr': 0.02},
                {'params': biases, 'use_fallback': True, 'lr': 3e-4}, # use AdamW if use_fallback=True
            ])
            ```
        """
        _validate_scalar("learning rate", lr)
        _validate_scalar("weight_decay", weight_decay)
        _validate_scalar("momentum", momentum, max_value=1.0)
        _validate_scalar("epsilon", eps)
        if conv_mode not in ["flatten", "batched"]:
            raise ValueError(f"Invalid conv_mode: {conv_mode}")
        if algo not in ["muon", "adamuon"]:
            raise ValueError(f"Invalid algo: {algo}. Must be 'muon' or 'adamuon'")

        if adamw_lr is not None:
            warnings.warn(
                "adamw_lr is deprecated, use fallback_lr_scale=adamw_lr/lr instead. "
                "adamw_lr will be removed in a future release.",
                FutureWarning,
                stacklevel=2,
            )
            if torch.is_tensor(lr):
                raise ValueError("adamw_lr is not supported with tensor lr; use fallback_lr_scale instead.")
            if lr == 0:
                raise ValueError("Cannot compute fallback_lr_scale from adamw_lr when lr=0")
            fallback_lr_scale = adamw_lr / lr

        defaults = dict(
            lr=lr,
            weight_decay=weight_decay,
            momentum=momentum,
            nesterov=nesterov,
            ns_steps=ns_steps,
            ns_coefficients=ns_coefficients,

View on GitHub (pinned to 9a5261e31b)

Solutions

1. Replace adamw_lr=X with fallback_lr_scale=X/lr (compute the ratio)
2. Keep lr itself unchanged so effective fallback LR stays the same
3. Update any YAML/CLI configs that pass adamw_lr

Example fix

# before
Muon(model.parameters(), lr=0.02, adamw_lr=0.0035)
# after
Muon(model.parameters(), lr=0.02, fallback_lr_scale=0.0035/0.02)

adamw_lr, lr = 0.0035, 0.02
assert adamw_lr is None, 'migrate: pass fallback_lr_scale=adamw_lr/lr'
opt = Muon(params, lr=lr, fallback_lr_scale=adamw_lr / lr)

When it happens

Trigger: timm.optim.Muon(..., adamw_lr=0.0035) or create_optimizer_v2(..., opt='muon', adamw_lr=...) with any non-None adamw_lr.

Common situations: Configs copied from modded-nanogpt style Muon recipes (e.g. adamw_lr=3.5e-3 with lr=0.02) after timm changed the API.

Related errors

• adamw_lr is not supported with tensor lr; use fallback_lr_sc
• Cannot compute fallback_lr_scale from adamw_lr when lr=0
• {name} must be a scalar or scalar tensor.
• Invalid {name}: {value}
• Tensor must have at least 2 dimensions, got {tensor.ndim}

AI-assisted analysis of huggingface/pytorch-image-models@9a5261e31b (2026-08-27). Data as JSON: /api/errors/81282ae979eab9a1. Report an issue: GitHub.