hiyouga/LlamaFactory · error · ValueError

Please specify `max_steps` in streaming mode.

Error message

Please specify `max_steps` in streaming mode.

What it means

Raised in parser.py:475 when training_args.max_steps == -1 (the default: rely on epochs) and data_args.streaming is true. Streaming datasets have no known length, so epoch-based training loops cannot compute an end condition.

Source

Thrown at src/llamafactory/hparams/parser.py:475

        if finetuning_args.reward_model_type == "lora" and model_args.use_kt:
            raise ValueError("KTransformers does not support lora reward model.")

        if finetuning_args.reward_model_type == "lora" and model_args.use_unsloth:
            raise ValueError("Unsloth does not support lora reward model.")

        if training_args.report_to and any(
            logger not in ("wandb", "tensorboard", "trackio", "none") for logger in training_args.report_to
        ):
            raise ValueError("PPO only accepts wandb, tensorboard, or trackio logger.")

    if not model_args.use_kt and training_args.parallel_mode == ParallelMode.NOT_DISTRIBUTED:
        raise ValueError("Please launch distributed training with `llamafactory-cli` or `torchrun`.")

    if training_args.deepspeed and training_args.parallel_mode != ParallelMode.DISTRIBUTED:
        raise ValueError("Please use `FORCE_TORCHRUN=1` to launch DeepSpeed training.")

    if training_args.max_steps == -1 and data_args.streaming:
        raise ValueError("Please specify `max_steps` in streaming mode.")

    if training_args.do_train and data_args.dataset is None:
        raise ValueError("Please specify dataset for training.")

    if (training_args.do_eval or training_args.do_predict or training_args.predict_with_generate) and (
        data_args.eval_dataset is None and data_args.val_size < 1e-6
    ):
        raise ValueError("Please make sure eval_dataset be provided or val_size >1e-6")

    if training_args.predict_with_generate:
        if is_deepspeed_zero3_enabled():
            raise ValueError("`predict_with_generate` is incompatible with DeepSpeed ZeRO-3.")

        if finetuning_args.compute_accuracy:
            raise ValueError("Cannot use `predict_with_generate` and `compute_accuracy` together.")

    if training_args.do_train and model_args.quantization_device_map == "auto":
        raise ValueError("Cannot use device map for quantized models in training.")

View on GitHub (pinned to f28afaf635)

Solutions

1. Add an explicit `max_steps: 1000` (or your budget) to the training config
2. Alternatively remove `streaming: true` if the dataset fits on disk, then use num_train_epochs

Example fix

# before (YAML)
streaming: true
# max_steps not set (defaults to -1)

# after
streaming: true
max_steps: 1000

if config.get("streaming") and config.get("max_steps", -1) == -1:
    raise SystemExit("streaming: true requires an explicit max_steps")

When it happens

Trigger: A YAML/JSON config containing `streaming: true` without a `max_steps` entry, so max_steps keeps its default -1; equally `--streaming` on the CLI with no --max_steps.

Common situations: Switching a large pretrain/SFT dataset to streaming mode to avoid disk usage and forgetting to add max_steps; copying a non-streaming example config and flipping only the streaming flag.

Related errors

• The model does not have a submodule named '{submodule_name}'
• Turn off `streaming` when saving dataset to disk.
• Please specify dataset for training.
• Please make sure eval_dataset be provided or val_size >1e-6
• Iterable dataset is not supported yet.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/b6dea30a38cffc0c. Report an issue: GitHub.