Lightning-AI/pytorch-lightning · error · NotImplementedError

All datasets are iterable-style datasets.

Error message

All datasets are iterable-style datasets.

What it means

CombinedLoader._dataset_length computes total dataset length by summing/reducing len() of the underlying datasets; if every dataset is iterable-style (no __len__), it cannot compute a length and raises NotImplementedError.

Source

Thrown at src/lightning/pytorch/utilities/combined_loader.py:374

        """Compute the number of batches."""
        if self._iterator is None:
            raise RuntimeError("Please call `iter(combined_loader)` first.")
        return len(self._iterator)

    def reset(self) -> None:
        """Reset the state and shutdown any workers."""
        if self._iterator is not None:
            self._iterator.reset()
            self._iterator = None
        for iterable in self.flattened:
            _shutdown_workers_and_reset_iterator(iterable)

    def _dataset_length(self) -> int:
        """Compute the total length of the datasets according to the current mode."""
        datasets = [getattr(dl, "dataset", None) for dl in self.flattened]
        lengths = [length for ds in datasets if (length := sized_len(ds)) is not None]
        if not lengths:
            raise NotImplementedError("All datasets are iterable-style datasets.")
        fn = _SUPPORTED_MODES[self._mode]["fn"]
        return fn(lengths)

    def _state_dicts(self) -> list[dict[str, Any]]:
        """Returns the list of state dicts for iterables in `self.flattened` that are stateful."""
        return [loader.state_dict() for loader in self.flattened if isinstance(loader, _Stateful)]

    def _load_state_dicts(self, states: list[dict[str, Any]]) -> None:
        """Loads the state dicts for iterables in `self.flattened` that are stateful."""
        if not states:
            return
        stateful_loaders = [loader for loader in self.flattened if isinstance(loader, _Stateful)]
        if len(stateful_loaders) != len(states):
            raise RuntimeError(
                f"The CombinedLoader has {len(stateful_loaders)} stateful loaders, but found {len(states)} states"
                " in the checkpoint. Please make sure you define the same dataloaders that were used when saving"
                " the checkpoint."
            )

View on GitHub (pinned to 9fed5c27d2)

Solutions

1. Use map-style datasets with __len__ where a length is required
2. Avoid features that need total length (adjust val_check_interval / limits so length isn't queried)
3. Wrap iterables in a dataset exposing an approximate __len__

Example fix

# before
cl = CombinedLoader([DataLoader(IterableDS()), DataLoader(IterableDS())])
len(cl._dataset_length())  # NotImplementedError path
# after
class SizedIterable(IterableDS):
    def __len__(self): return 1000
cl = CombinedLoader([DataLoader(SizedIterable()), DataLoader(SizedIterable())])

from lightning.pytorch.utilities.data import sized_len
def has_sized_dataset(cl) -> bool:
    return any(sized_len(getattr(dl, "dataset", None)) is not None for dl in cl.flattened)

def loader_has_length(dl) -> bool:
    from lightning.pytorch.utilities.data import sized_len
    return sized_len(getattr(dl, "dataset", None)) is not None

try:
    total = cl._dataset_length()
except NotImplementedError:
    total = None  # streaming mode; skip length-dependent features

When it happens

Trigger: Building CombinedLoader from IterableDataset-based dataloaders and triggering length computation (e.g. progress bar sizing, limit_percent or checkpoint-length logic that needs a length).

Common situations: Streaming datasets, TFRecord/JSON-lines iterable datasets, WebDataset-style pipelines inside a combined loader.

Related errors

• The `CSVLogger` does not yet support logging hyperparameters
• DeepSpeed handles gradient clipping automatically within the
• The `{type(self).__name__}` does not use the `CheckpointIO`
• The `{type(self).__name__}` does not support setting a `Chec
• Loading a single optimizer object from a checkpoint is not s

AI-assisted analysis of Lightning-AI/pytorch-lightning@9fed5c27d2 (2026-08-28). Data as JSON: /api/errors/2b1050988e821742. Report an issue: GitHub.