Lightning-AI/pytorch-lightning · error · TypeError

You seem to have configured a sampler in your DataLoader whi

Error message

You seem to have configured a sampler in your DataLoader which does not provide `__len__` method. The sampler was about to be replaced by `DistributedSamplerWrapper` since `use_distributed_sampler` is True and you are using distributed training. Either provide `__len__` method in your sampler, remove it from DataLoader or set `use_distributed_sampler=False` if you want to handle distributed sampling yourself.

What it means

When use_distributed_sampler=True (the default) and you supply a custom sampler to your DataLoader, Lightning wraps it in DistributedSamplerWrapper backed by _DatasetSamplerWrapper, which needs the sampler's length. If the sampler isn't collections.abc.Sized (no __len__), TypeError is raised telling you to add __len__, drop the sampler, or disable the wrapping.

Source

Thrown at src/lightning/fabric/utilities/distributed.py:318

    if _distributed_is_initialized():
        torch.distributed.destroy_process_group()
    signal.signal(signal.SIGINT, signal.SIG_DFL)


def _get_default_process_group_backend_for_device(device: torch.device) -> str:
    """Return corresponding distributed backend for a given device."""
    device_backend_map = torch.distributed.Backend.default_device_backend_map
    if device.type in device_backend_map:
        return device_backend_map[device.type]
    return "gloo"


class _DatasetSamplerWrapper(Dataset):
    """Dataset to create indexes from `Sampler` or `Iterable`"""

    def __init__(self, sampler: Union[Sampler, Iterable]) -> None:
        if not isinstance(sampler, Sized):
            raise TypeError(
                "You seem to have configured a sampler in your DataLoader which"
                " does not provide `__len__` method. The sampler was about to be"
                " replaced by `DistributedSamplerWrapper` since `use_distributed_sampler`"
                " is True and you are using distributed training. Either provide `__len__`"
                " method in your sampler, remove it from DataLoader or set `use_distributed_sampler=False`"
                " if you want to handle distributed sampling yourself."
            )
        if len(sampler) == float("inf"):
            raise TypeError(
                "You seem to have configured a sampler in your DataLoader which"
                " does not provide finite `__len__` method. The sampler was about to be"
                " replaced by `DistributedSamplerWrapper` since `use_distributed_sampler`"
                " is True and you are using distributed training. Either provide `__len__`"
                " method in your sampler which returns a finite number, remove it from DataLoader"
                " or set `use_distributed_sampler=False` if you want to handle distributed sampling yourself."
            )
        self._sampler = sampler
        # defer materializing an iterator until it is necessary

View on GitHub (pinned to 9fed5c27d2)

Solutions

1. Add def __len__(self) returning the number of samples to your sampler
2. Or set use_distributed_sampler=False in DataLoader kwargs (Fabric(...setup_dataloaders(dl, use_distributed_sampler=False))) and shard manually
3. Or remove the sampler and let Lightning's default DistributedSampler handle sharding

Example fix

# before
class MySampler(Sampler):
    def __iter__(self): ...

dl = DataLoader(ds, sampler=MySampler())
fabric.setup_dataloaders(dl)

# after
class MySampler(Sampler):
    def __iter__(self): ...
    def __len__(self):
        return len(self.data_source)

dl = DataLoader(ds, sampler=MySampler())
fabric.setup_dataloaders(dl)

from collections.abc import Sized

if sampler is not None and not isinstance(sampler, Sized):
    raise ValueError("sampler needs __len__; or pass use_distributed_sampler=False")

def sampler_is_sized(s) -> bool:
    return isinstance(s, Sized) and len(s) != float("inf")

When it happens

Trigger: Passing an iterable-style/streams sampler (e.g. torch.utils.data.IterableSampler-like or a custom iterator without __len__) as DataLoader(dataset, sampler=...) with Fabric/Trainer distributed training and default use_distributed_sampler=True.

Common situations: Streaming/infinite datasets with custom samplers; migrating single-GPU code that never needed __len__; using BatchSampler or weight-jittered samplers that skip __len__.

Related errors

• You seem to have configured a sampler in your DataLoader whi
• The dataloader {dataloader} needs to subclass `torch.utils.d
• Trying to inject custom `Sampler` into the `{dataloader_cls_
• f"An invalid dataloader was passed to `Trainer.{trainer_fn.v
• f"An invalid dataloader was passed to `Trainer.{trainer_fn.v

AI-assisted analysis of Lightning-AI/pytorch-lightning@9fed5c27d2 (2026-08-28). Data as JSON: /api/errors/44d0ac2cee4be372. Report an issue: GitHub.