huggingface/pytorch-image-models · error · ValueError

{name} must be a scalar or scalar tensor.

Error message

{name} must be a scalar or scalar tensor.

What it means

timm optimizer helper _validate_scalar accepts a python number or a single-element tensor for hyper-parameters like lr/beta/eps. A tensor with more than one element cannot be interpreted as a scalar hyper-parameter.

Source

Thrown at timm/optim/_helpers.py:71

    return capturable_supported_devices


def _check_capturable_devices(
        params: Sequence[Tensor],
        state_steps: Sequence[Tensor],
        supports_xla: bool = True,
) -> None:
    capturable_supported_devices = _get_capturable_supported_devices(supports_xla=supports_xla)
    assert all(
        p.device.type == step.device.type and p.device.type in capturable_supported_devices
        for p, step in zip(params, state_steps)
    ), f"If capturable=True, params and state_steps must be on supported devices: {capturable_supported_devices}."


def _validate_scalar(name: str, value, min_value: float = 0.0, max_value: Optional[float] = None) -> None:
    if torch.is_tensor(value):
        if value.numel() != 1:
            raise ValueError(f"{name} must be a scalar or scalar tensor.")
        value_float = float(value.detach().cpu())
    else:
        value_float = float(value)
    if value_float < min_value or (max_value is not None and value_float >= max_value):
        raise ValueError(f"Invalid {name}: {value}")


def _add_scaled_(param: Tensor, update: Tensor, scale) -> None:
    if torch.is_tensor(scale):
        param.add_(update * scale)
    else:
        param.add_(update, alpha=scale)


def _addcdiv_scaled_(param: Tensor, tensor1: Tensor, tensor2: Tensor, scale) -> None:
    if torch.is_tensor(scale):
        param.add_(tensor1 / tensor2 * scale)
    else:

View on GitHub (pinned to 9a5261e31b)

Solutions

1. Pass plain floats (lr=1e-3, betas=(0.9, 0.999))
2. If a tensor arrives from elsewhere, extract a scalar: float(t) or t.item() after asserting t.numel()==1

Example fix

# before
opt = timm.optim.create_optimizer_v2(model, opt='adamw', lr=torch.tensor([1e-3]))
# after
opt = timm.optim.create_optimizer_v2(model, opt='adamw', lr=1e-3)

hp = float(hp_tensor.item()) if torch.is_tensor(hp_tensor) else float(hp)
assert not torch.is_tensor(hp_tensor) or hp_tensor.numel() == 1

def as_scalar_hp(v):
    if torch.is_tensor(v):
        assert v.numel() == 1, 'hyper-parameter must be scalar'
        return v.item()
    return float(v)

try:
    opt = timm.optim.AdamW(params, lr=lr, betas=betas)
except ValueError as e:
    if 'must be a scalar' in str(e):
        opt = timm.optim.AdamW(params, lr=float(lr.item()), betas=(float(betas[0].item()), float(betas[1].item())))
    else:
        raise

When it happens

Trigger: Passing a multi-element tensor as an optimizer hyper-parameter, e.g. AdamW(model.parameters(), lr=torch.tensor([1e-3, 1e-4])) or a per-group param tensor passed where a scalar is expected.

Common situations: Programmatically building hyper-parameters as tensors (e.g. slices of arrays) instead of floats; migrating code that accidentally passes a shape-(1,1) or vector tensor.

Understand the failure class

Background: Invalid argument type errors: "must be of type string", "expected X, got Y", and ERR_INVALID_ARG_TYPE explained — this error's family across 15 libraries.

Related errors

• Invalid {name}: {value}
• It is highly recommended to have 'opt_einsum' installed for
• adamw_lr is deprecated, use fallback_lr_scale=adamw_lr/lr in
• Input image must have positive dimensions, got H={height}, W
• Invalid class map file, expected a dict ({class_map_path}).

AI-assisted analysis of huggingface/pytorch-image-models@9a5261e31b (2026-08-27). Data as JSON: /api/errors/76d34cc0345a2ff5. Report an issue: GitHub.