Lightning-AI/pytorch-lightning · error · ValueError

Tuning the batch size is currently not supported with distri

Error message

Tuning the batch size is currently not supported with distributed strategies.

What it means

Batch-size scaling (tuner.scale_batch_size) is not supported under distributed strategies (DDP, etc.) because it works by repeatedly re-fitting with growing batch sizes. _check_scale_batch_size_configuration raises ValueError when trainer._accelerator_connector.is_distributed is true.

Source

Thrown at src/lightning/pytorch/tuner/tuning.py:243

                " arguments should be None, please consider setting `dataloaders` instead."
            )


def _check_lr_find_configuration(trainer: "pl.Trainer") -> None:
    # local import to avoid circular import
    from lightning.pytorch.callbacks.lr_finder import LearningRateFinder

    configured_callbacks = [cb for cb in trainer.callbacks if isinstance(cb, LearningRateFinder)]
    if configured_callbacks:
        raise ValueError(
            "Trainer is already configured with a `LearningRateFinder` callback."
            "Please remove it if you want to use the Tuner."
        )


def _check_scale_batch_size_configuration(trainer: "pl.Trainer") -> None:
    if trainer._accelerator_connector.is_distributed:
        raise ValueError("Tuning the batch size is currently not supported with distributed strategies.")

    # local import to avoid circular import
    from lightning.pytorch.callbacks.batch_size_finder import BatchSizeFinder

    configured_callbacks = [cb for cb in trainer.callbacks if isinstance(cb, BatchSizeFinder)]
    if configured_callbacks:
        raise ValueError(
            "Trainer is already configured with a `BatchSizeFinder` callback."
            "Please remove it if you want to use the Tuner."
        )

View on GitHub (pinned to 9fed5c27d2)

Solutions

1. Run scale_batch_size on a single device (strategy='auto'/None, devices=1) to find a per-device batch size, then scale manually for distributed runs
2. Use the BatchSizeFinder callback alternative only if it fits your non-distributed setup
3. Tune on a smaller reproducible config before launching distributed training

Example fix

# before
trainer = Trainer(accelerator="gpu", devices=4, strategy="ddp")
trainer.tuner.scale_batch_size(model)
# after
tuner_trainer = Trainer(accelerator="gpu", devices=1)
model.hparams.batch_size = tuner_trainer.tuner.scale_batch_size(model)
trainer = Trainer(accelerator="gpu", devices=4, strategy="ddp")

assert not trainer._accelerator_connector.is_distributed, "scale_batch_size requires a non-distributed setup"

When it happens

Trigger: trainer = Trainer(strategy='ddp', devices=2, accelerator='gpu') then trainer.tuner.scale_batch_size(model); also multi-process launches (torchrun) or strategies that imply distribution.

Common situations: Tuning batch size in a multi-GPU or multi-node setup; forgetting devices>1 implies distributed.

Related errors

• To use Fabric with more than one device, you must call `.lau
• Trying to inject a modified sampler into the batch sampler;
• Skipping the `training_step` by returning None in distribute
• Post-localSGD algorithm is used, but model averaging period
• Trainer is already configured with a `BatchSizeFinder` callb

AI-assisted analysis of Lightning-AI/pytorch-lightning@9fed5c27d2 (2026-08-28). Data as JSON: /api/errors/32572e2da08c2049. Report an issue: GitHub.