Lightning-AI/pytorch-lightning · error · MisconfigurationException

The `swa_lrs` should a positive float, or a list of positive

Error message

The `swa_lrs` should a positive float, or a list of positive floats

What it means

StochasticWeightAveraging requires swa_lrs (the constant learning rate used during SWA) to be a positive float or a list of positive floats (one per optimizer param group / one per optimizer). This MisconfigurationException fires when the value has the wrong type, is non-positive, or the list contains non-positive/non-float entries.

Source

Thrown at src/lightning/pytorch/callbacks/stochastic_weight_avg.py:110

                equally weighted average is used (default: ``None``)

            device: if provided, the averaged model will be stored on the ``device``.
                When None is provided, it will infer the `device` from ``pl_module``.
                (default: ``"cpu"``)

        """

        err_msg = "swa_epoch_start should be a >0 integer or a float between 0 and 1."
        if isinstance(swa_epoch_start, int) and swa_epoch_start < 1:
            raise MisconfigurationException(err_msg)
        if isinstance(swa_epoch_start, float) and not (0 <= swa_epoch_start <= 1):
            raise MisconfigurationException(err_msg)

        wrong_type = not isinstance(swa_lrs, (float, list))
        wrong_float = isinstance(swa_lrs, float) and swa_lrs <= 0
        wrong_list = isinstance(swa_lrs, list) and not all(lr > 0 and isinstance(lr, float) for lr in swa_lrs)
        if wrong_type or wrong_float or wrong_list:
            raise MisconfigurationException("The `swa_lrs` should a positive float, or a list of positive floats")

        if avg_fn is not None and not callable(avg_fn):
            raise MisconfigurationException("The `avg_fn` should be callable.")

        if device is not None and not isinstance(device, (torch.device, str)):
            raise MisconfigurationException(f"device is expected to be a torch.device or a str. Found {device}")

        self.n_averaged: Optional[Tensor] = None
        self._swa_epoch_start = swa_epoch_start
        self._swa_lrs = swa_lrs
        self._annealing_epochs = annealing_epochs
        self._annealing_strategy = annealing_strategy
        self._avg_fn = avg_fn or self.avg_fn
        self._device = device
        self._model_contains_batch_norm: Optional[bool] = None
        self._average_model: Optional[pl.LightningModule] = None
        self._initialized = False
        self._swa_scheduler: Optional[LRScheduler] = None

View on GitHub (pinned to 9fed5c27d2)

Solutions

1. Pass a positive float, e.g. swa_lrs=1e-3
2. For multiple param groups pass a list of positive floats matching them: swa_lrs=[1e-3, 1e-4]
3. Ensure list elements are Python floats, not ints or strings

Example fix

# before
swa = SWA(swa_lrs=0)  # or 1 (int)
# after
swa = SWA(swa_lrs=1e-3)  # or [1e-3, 1e-4] for two param groups

def valid_swa_lrs(v):
    if isinstance(v, float):
        return v > 0
    if isinstance(v, list):
        return all(type(x) is float and x > 0 for x in v)
    return False
assert valid_swa_lrs(cfg.swa_lrs)

def is_valid_swa_lrs(v) -> bool:
    return (type(v) is float and v > 0) or (isinstance(v, list) and v and all(type(x) is float and x > 0 for x in v))

When it happens

Trigger: SWA(swa_lrs=-0.1), SWA(swa_lrs=0), SWA(swa_lrs='1e-3') (string), or SWA(swa_lrs=[0.01, 0]) with multiple param groups.

Common situations: Reusing the peak LR scheduler value as swa_lrs when it decays to 0; passing ints like swa_lrs=1 (isinstance(1, float) is False -> wrong_type); building the list programmatically and including an int element.

Related errors

• swa_epoch_start should be a >0 integer or a float between 0
• The `avg_fn` should be callable.
• device is expected to be a torch.device or a str. Found {dev
• Device should be CUDA, got {device} instead.
• You requested to find {num_devices} devices but there are no

AI-assisted analysis of Lightning-AI/pytorch-lightning@9fed5c27d2 (2026-08-28). Data as JSON: /api/errors/c42986634e765ab3. Report an issue: GitHub.