unslothai/unsloth · error · HTTPException

dataset_streaming requires max_steps > 0 because streaming d

Error message

dataset_streaming requires max_steps > 0 because streaming datasets have no known length.

What it means

HTTP 422 when dataset_streaming=true and max_steps is None or <= 0: a streaming dataset has no known length, so epoch-based stopping cannot work and an explicit step budget is mandatory.

Source

Thrown at studio/backend/routes/training.py:1347

                    detail = "dataset_streaming requires hf_dataset; streaming is not supported for local datasets.",
                )
            if request.is_dataset_image or request.is_dataset_audio:
                raise HTTPException(
                    status_code = 400,
                    detail = "dataset_streaming is not supported for vision or audio datasets.",
                )
            if request.is_embedding:
                raise HTTPException(
                    status_code = 400,
                    detail = "dataset_streaming is not supported for embedding training; the embedding loader needs the full dataset.",
                )
            if _hw.DEVICE == _hw.DeviceType.MLX:
                raise HTTPException(
                    status_code = 400,
                    detail = "dataset_streaming is not yet supported on Apple Silicon (MLX); the MLX loader materializes the full dataset.",
                )
            if request.max_steps is None or request.max_steps <= 0:
                raise HTTPException(
                    status_code = 422,
                    detail = "dataset_streaming requires max_steps > 0 because streaming datasets have no known length.",
                )
            if request.train_on_completions:
                raise HTTPException(
                    status_code = 422,
                    detail = "dataset_streaming is not supported with train_on_completions yet.",
                )
            if request.eval_steps > 0:
                train_split = request.train_split or "train"
                if not request.eval_split or request.eval_split == train_split:
                    raise HTTPException(
                        status_code = 422,
                        detail = "dataset_streaming with evaluation requires a separate eval_split.",
                    )
            # Streaming is HF-only: reject when the request also carries a local dataset path or an
            # S3 config, since those sources cannot be streamed via HF's loader.
            if request.local_datasets:

View on GitHub (pinned to 203007d190)

Solutions

1. Set max_steps to a positive integer equal to the desired training budget (steps = epochs x approx_stream_length)
2. Or disable streaming so length-based epoch training applies
3. Sanity-check max_steps against your desired effective epoch count for the streamed data

Example fix

// before
{"dataset_streaming": true, "hf_dataset": "org/ds"}  // 422

// after
{"dataset_streaming": true, "hf_dataset": "org/ds", "max_steps": 5000}

def streaming_config_valid(p: dict) -> bool:
    if not p.get("dataset_streaming"):
        return True
    ms = p.get("max_steps")
    return isinstance(ms, int) and ms > 0

resp = client.post("/training/start", payload)
if resp.status_code == 422 and "max_steps" in resp.text:
    payload["max_steps"] = estimate_steps(desired_epochs, approx_stream_length)
    resp = client.post("/training/start", payload)

When it happens

Trigger: POST /training/start with dataset_streaming: true without a positive max_steps (relying on num_train_epochs / default stopping).

Common situations: Porting a config that used epochs on a bounded dataset to a streamed dataset; omitting max_steps because non-streaming runs default sensibly.

Related errors

• dataset_streaming requires hf_dataset; streaming is not supp
• dataset_streaming is not supported with train_on_completions
• dataset_streaming with evaluation requires a separate eval_s
• gradient_accumulation_steps must be >= 1
• lora_rank must be >= 1

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/4a5ab17c4c4d3332. Report an issue: GitHub.