unslothai/unsloth · error · HTTPException

dataset_streaming is HF-only; remove local_datasets / S3 sou

Error message

dataset_streaming is HF-only; remove local_datasets / S3 source. Streaming is not supported with S3 datasets.

What it means

HTTP 400 when dataset_streaming=true and s3_config is set: streaming is HF-Hub-only, and datasets staged from S3 are downloaded files that the HF streaming loader cannot consume. S3 sources and streaming are mutually exclusive.

Source

Thrown at studio/backend/routes/training.py:1374

            if request.eval_steps > 0:
                train_split = request.train_split or "train"
                if not request.eval_split or request.eval_split == train_split:
                    raise HTTPException(
                        status_code = 422,
                        detail = "dataset_streaming with evaluation requires a separate eval_split.",
                    )
            # Streaming is HF-only: reject when the request also carries a local dataset path or an
            # S3 config, since those sources cannot be streamed via HF's loader.
            if request.local_datasets:
                raise HTTPException(
                    status_code = 400,
                    detail = (
                        "dataset_streaming is HF-only; remove local_datasets / S3 source. "
                        "Streaming is not supported with local file paths."
                    ),
                )
            if request.s3_config is not None:
                raise HTTPException(
                    status_code = 400,
                    detail = (
                        "dataset_streaming is HF-only; remove local_datasets / S3 source. "
                        "Streaming is not supported with S3 datasets."
                    ),
                )
            if request.dataset_known_cached or request.dataset_local_path:
                raise HTTPException(
                    status_code = 422,
                    detail = (
                        "dataset_streaming streams from the Hub and cannot use the local "
                        "dataset cache; disable streaming to train from the cached copy."
                    ),
                )
        model_preflight = await asyncio.to_thread(
            _reject_untrainable_model_request,
            request,
            resume_actual_model_repo_id,

View on GitHub (pinned to 203007d190)

Solutions

Set s3_config to null/omit it and provide hf_dataset when streaming
Or disable dataset_streaming to keep training from the S3 dataset
Ensure your client does not send an empty s3_config object - the field must be null, not {}

Example fix

// before
{"dataset_streaming": true, "s3_config": {"bucket": "data", "key": "ds.jsonl"}}  // 400

// after
{"dataset_streaming": false, "s3_config": {"bucket": "data", "key": "ds.jsonl"}}
// or
{"dataset_streaming": true, "hf_dataset": "org/ds"}

Defensive patterns

Strategy: validation

Validate before calling

def streaming_config_valid(p: dict) -> bool:
    if not p.get("dataset_streaming"):
        return True
    return p.get("s3_config") is None

Try / catch

resp = client.post("/training/start", payload)
if resp.status_code == 400 and "S3" in resp.text:
    payload["s3_config"] = None
    payload["hf_dataset"] = "org/ds"  # streaming needs a Hub dataset
    resp = client.post("/training/start", payload)

Prevention

Omit s3_config entirely (or send null) when dataset_streaming is true
Make sure your client SDK does not serialize an empty s3_config object by default
Pair streaming exclusively with an hf_dataset id

When it happens

Trigger: POST /training/start with dataset_streaming: true and s3_config != null.

Common situations: Switching a pipeline from S3-hosted data to a Hub dataset but leaving s3_config in the request payload; client SDK that always serializes an (empty) s3_config object.

Related errors

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/9e5d414772622f16. Report an issue: GitHub.