hiyouga/LlamaFactory · error · ValueError

Streaming dataset does not support index access.

Error message

Streaming dataset does not support index access.

What it means

The v1 `DataEngine.__getitem__` raises immediately when `self.streaming` is true, because index-based random access requires a materialized `data_index`, and streaming `IterableDataset`s only support sequential iteration. This blocks DataLoader workers and any selection plugin that assumes positional indexing.

Source

Thrown at src/llamafactory/v1/core/data_engine.py:182

        Returns:
            int: Dataset length.
        """
        if self.streaming:
            return -1
        else:
            return len(self.data_index)

    def __getitem__(self, index: int | Any) -> Sample | list[Sample]:
        """Get dataset item.

        Args:
            index (int): Dataset index.

        Returns:
            Sample: Dataset item.
        """
        if self.streaming:
            raise ValueError("Streaming dataset does not support index access.")

        if isinstance(index, int):
            return self._get(*self.data_index[index])
        else:  # data selector plugin
            from ..plugins.data_plugins.loader import select_data_sample

            selected_index = select_data_sample(self.data_index, index)
            if isinstance(selected_index, list):
                return [self._get(*entry) for entry in selected_index]
            else:
                return self._get(*selected_index)

    def _get(self, dataset_name: str, sample_index: int, cut: int | None = None) -> Sample:
        """Convert one raw row, truncating to a multi-turn prefix when ``cut`` is set."""
        sample = self._convert_data_sample(self.datasets[dataset_name][sample_index], dataset_name)
        if cut is not None and "messages" in sample:
            sample = {**sample, "messages": sample["messages"][:cut]}
        return sample

View on GitHub (pinned to f28afaf635)

Solutions

1. Remove `streaming: true` so the dataset is materialized and indexable (ensure disk for the download)
2. Keep streaming and iterate sequentially instead of indexing (e.g. iterate the engine's stream, use `.take()/.skip()` style ops on the IterableDataset)
3. For evaluation on a subset, materialize only the eval split locally and stream only the train split in a separate engine

Example fix

# before
for i in range(len(dataset)):   # or dataset[0]
    sample = dataset[i]          # ValueError: streaming

# after (non-streaming)
dataset_info:
  mydata:
    path: org/data
    # streaming: true  <- removed
sample = dataset[0]

def assert_indexable(engine) -> None:
    if getattr(engine, "streaming", False):
        raise SystemExit("engine is streaming; index access unsupported — materialize the dataset first")

from llamafactory.v1.core.data_engine import DataEngine

def is_indexable(engine: DataEngine) -> bool:
    return not getattr(engine, "streaming", False)

try:
    sample = engine[i]
except ValueError as e:
    if "Streaming dataset" in str(e):
        # fall back to sequential iteration for streaming engines
        it = iter(engine)  # or engine.iterate()
    else:
        raise

When it happens

Trigger: Any code path that calls `dataset[i]` (dataloader with a sampler, a `select(...)` data-selector plugin, or manual probing) while the DataEngine was constructed with one or more streaming datasets.

Common situations: Enabling `streaming: true` for a huge hub dataset but keeping a training loop or evaluation code that samples by index; using shuffling/eval splits that require `len()` and random access.

Related errors

• All datasets must be streaming or non-streaming.
• dim must be specified.
• Iterable dataset is not supported yet.
• The model does not have a submodule named '{submodule_name}'
• Cannot stream function calls.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/51d5da3c22460d56. Report an issue: GitHub.