Lightning-AI/pytorch-lightning · error · MisconfigurationException

swa_epoch_start should be a >0 integer or a float between 0

Error message

swa_epoch_start should be a >0 integer or a float between 0 and 1.

What it means

StochasticWeightAveraging validates swa_epoch_start in __init__: if passed as an int it must be >=1 (an epoch number), and if passed as a float it must be in [0,1] (fraction of training). This branch fires for an int < 1, e.g. 0 or negative.

Source

Thrown at src/lightning/pytorch/callbacks/stochastic_weight_avg.py:102

                - ``"cos"``. For cosine annealing.
                - ``"linear"`` For linear annealing

            avg_fn: the averaging function used to update the parameters;
                the function must take in the current value of the
                :class:`AveragedModel` parameter, the current value of :attr:`model`
                parameter and the number of models already averaged; if None,
                equally weighted average is used (default: ``None``)

            device: if provided, the averaged model will be stored on the ``device``.
                When None is provided, it will infer the `device` from ``pl_module``.
                (default: ``"cpu"``)

        """

        err_msg = "swa_epoch_start should be a >0 integer or a float between 0 and 1."
        if isinstance(swa_epoch_start, int) and swa_epoch_start < 1:
            raise MisconfigurationException(err_msg)
        if isinstance(swa_epoch_start, float) and not (0 <= swa_epoch_start <= 1):
            raise MisconfigurationException(err_msg)

        wrong_type = not isinstance(swa_lrs, (float, list))
        wrong_float = isinstance(swa_lrs, float) and swa_lrs <= 0
        wrong_list = isinstance(swa_lrs, list) and not all(lr > 0 and isinstance(lr, float) for lr in swa_lrs)
        if wrong_type or wrong_float or wrong_list:
            raise MisconfigurationException("The `swa_lrs` should a positive float, or a list of positive floats")

        if avg_fn is not None and not callable(avg_fn):
            raise MisconfigurationException("The `avg_fn` should be callable.")

        if device is not None and not isinstance(device, (torch.device, str)):
            raise MisconfigurationException(f"device is expected to be a torch.device or a str. Found {device}")

        self.n_averaged: Optional[Tensor] = None
        self._swa_epoch_start = swa_epoch_start
        self._swa_lrs = swa_lrs

View on GitHub (pinned to 9fed5c27d2)

Solutions

1. Use swa_epoch_start=1 to start averaging from the first epoch
2. Or use a float fraction like 0.75 to start SWA in the last quarter of training

Example fix

# before
swa = SWA(swa_epoch_start=0)
# after
swa = SWA(swa_epoch_start=1)  # or SWA(swa_epoch_start=0.75)

def valid_swa_epoch_start(v):
    return (isinstance(v, int) and not isinstance(v, bool) and v >= 1) or (isinstance(v, float) and 0 <= v <= 1)
assert valid_swa_epoch_start(cfg.swa_epoch_start)

def is_valid_swa_start(v) -> bool:
    return (type(v) is int and v >= 1) or (type(v) is float and 0.0 <= v <= 1.0)

When it happens

Trigger: SWA(swa_epoch_start=0) or any integer less than 1 (e.g. -2).

Common situations: Assuming epoch numbering starts at 0; passing 0 intending 'start immediately'; copy-pasting a float default as int.

Related errors

• The `avg_fn` should be callable.
• device is expected to be a torch.device or a str. Found {dev
• The `swa_lrs` should a positive float, or a list of positive
• SWA currently works with 1 `optimizer`.
• SWA currently not supported for more than 1 `lr_scheduler`.

AI-assisted analysis of Lightning-AI/pytorch-lightning@9fed5c27d2 (2026-08-28). Data as JSON: /api/errors/453b7f232dc594f7. Report an issue: GitHub.