hiyouga/LlamaFactory · error · ValueError

Streaming mode should have an integer val size.

Error message

Streaming mode should have an integer val size.

What it means

LlamaFactory splits a fraction of the training set as validation when val_size is between 0 and 1, but HuggingFace IterableDatasets (streaming mode) cannot be randomly split by fraction. The check in DataArguments.__post_init__ (src/llamafactory/hparams/data_args.py:177) therefore rejects fractional val_size when streaming=True. Use an integer val_size, which takes a fixed number of samples from the stream.

Source

Thrown at src/llamafactory/hparams/data_args.py:177

        if self.dataset is None and self.val_size > 1e-6:
            raise ValueError("Cannot specify `val_size` if `dataset` is None.")

        if self.eval_dataset is not None and self.val_size > 1e-6:
            raise ValueError("Cannot specify `val_size` if `eval_dataset` is not None.")

        if self.interleave_probs is not None:
            if self.mix_strategy == "concat":
                raise ValueError("`interleave_probs` is only valid for interleaved mixing.")

            self.interleave_probs = list(map(float, split_arg(self.interleave_probs)))
            if self.dataset is not None and len(self.dataset) != len(self.interleave_probs):
                raise ValueError("The length of dataset and interleave probs should be identical.")

            if self.eval_dataset is not None and len(self.eval_dataset) != len(self.interleave_probs):
                raise ValueError("The length of eval dataset and interleave probs should be identical.")

        if self.streaming and self.val_size > 1e-6 and self.val_size < 1:
            raise ValueError("Streaming mode should have an integer val size.")

        if self.streaming and self.max_samples is not None:
            raise ValueError("`max_samples` is incompatible with `streaming`.")

        if self.mask_history and self.train_on_prompt:
            raise ValueError("`mask_history` is incompatible with `train_on_prompt`.")

        if self.neat_packing:
            self.packing = True

        if self.packing:
            self.cutoff_len -= 1  # avoid pad_to_multiple_of, needs improve

    def to_dict(self) -> dict[str, Any]:
        return asdict(self)

View on GitHub (pinned to f28afaf635)

Solutions

1. Set val_size to an integer number of samples, e.g. val_size: 500, in your training YAML.
2. Set val_size: 0 to disable the validation split entirely if you do not need eval during streaming training.
3. Disable streaming (streaming: false) if the dataset fits in memory and you want to keep a fractional split.

Example fix

# before (yaml)
streaming: true
val_size: 0.1

# after (yaml)
streaming: true
val_size: 500

def check_streaming_val_size(streaming: bool, val_size: float) -> None:
    if streaming and 1e-6 < val_size < 1:
        raise ValueError("Use an integer val_size with streaming=True, e.g. val_size=500")

try:
    data_args = DataArguments(dataset=..., streaming=True, val_size=0.1)
except ValueError as e:
    # fix config and re-run; this is a config error, never retry as-is
    raise SystemExit(f"Invalid data config: {e}")

When it happens

Trigger: A YAML/JSON training config (or DataArguments instance) with streaming: true and val_size set to a float in (0,1), e.g. val_size: 0.1. The condition self.streaming and self.val_size > 1e-6 and self.val_size < 1 fires during argument parsing, before any data is loaded.

Common situations: Copying a non-streaming config (where val_size: 0.1 is idiomatic) and flipping streaming: true for large datasets. Also happens when tuning large streamed corpora (e.g. pile, fineweb) where users keep the fractional split habit.

Related errors

• `max_samples` is incompatible with `streaming`.
• Turn off `streaming` when saving dataset to disk.
• The length of packed example should be identical to the cuto
• Cannot specify `val_size` if `dataset` is None.
• Cannot specify `val_size` if `eval_dataset` is not None.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/a97d0223d8be810b. Report an issue: GitHub.