hiyouga/LlamaFactory · error · ValueError

`max_samples` is incompatible with `streaming`.

Error message

`max_samples` is incompatible with `streaming`.

What it means

max_samples truncates a dataset by indexing the first N examples, which requires random access. Streaming datasets are IterableDatasets and cannot be indexed, so DataArguments.__post_init__ (src/llamafactory/hparams/data_args.py:180) forbids the combination. To limit a stream, use max_steps in the training arguments instead.

Source

Thrown at src/llamafactory/hparams/data_args.py:180

        if self.eval_dataset is not None and self.val_size > 1e-6:
            raise ValueError("Cannot specify `val_size` if `eval_dataset` is not None.")

        if self.interleave_probs is not None:
            if self.mix_strategy == "concat":
                raise ValueError("`interleave_probs` is only valid for interleaved mixing.")

            self.interleave_probs = list(map(float, split_arg(self.interleave_probs)))
            if self.dataset is not None and len(self.dataset) != len(self.interleave_probs):
                raise ValueError("The length of dataset and interleave probs should be identical.")

            if self.eval_dataset is not None and len(self.eval_dataset) != len(self.interleave_probs):
                raise ValueError("The length of eval dataset and interleave probs should be identical.")

        if self.streaming and self.val_size > 1e-6 and self.val_size < 1:
            raise ValueError("Streaming mode should have an integer val size.")

        if self.streaming and self.max_samples is not None:
            raise ValueError("`max_samples` is incompatible with `streaming`.")

        if self.mask_history and self.train_on_prompt:
            raise ValueError("`mask_history` is incompatible with `train_on_prompt`.")

        if self.neat_packing:
            self.packing = True

        if self.packing:
            self.cutoff_len -= 1  # avoid pad_to_multiple_of, needs improve

    def to_dict(self) -> dict[str, Any]:
        return asdict(self)

View on GitHub (pinned to f28afaf635)

Solutions

1. Remove max_samples from the config and limit the run with max_steps (or steps in the trainer) instead.
2. If you only need a small fixed dataset, disable streaming (streaming: false) and keep max_samples.
3. For debugging, use a small streamed dataset by name and stop early via max_steps / eval_steps.

Example fix

# before (yaml)
streaming: true
max_samples: 1000

# after (yaml)
streaming: true
# max_samples removed; limit training length instead
max_steps: 500

def check_streaming_max_samples(streaming: bool, max_samples: int | None) -> None:
    if streaming and max_samples is not None:
        raise ValueError("max_samples cannot be used with streaming=True; use max_steps instead")

try:
    data_args = DataArguments(dataset=..., streaming=True, max_samples=1000)
except ValueError as e:
    raise SystemExit(f"Invalid data config: {e}")

When it happens

Trigger: A training config with both streaming: true and max_samples: N (N any integer). Raised at argument-parsing time in DataArguments.__post_init__ before dataset construction.

Common situations: Users add max_samples: 1000 for a quick smoke-test run on a huge streamed dataset (common when testing a pipeline before a full run). Also occurs when reusing a debug config that had max_samples set and then enabling streaming for scale.

Related errors

• Streaming mode should have an integer val size.
• Cannot stream multiple responses.
• Turn off `streaming` when saving dataset to disk.
• The length of packed example should be identical to the cuto
• `mask_history` is incompatible with `train_on_prompt`.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/8626cd2e0119570d. Report an issue: GitHub.