{"record":{"id":"9e5d414772622f16","repo":"unslothai/unsloth","slug":"dataset-streaming-is-hf-only-remove-local-dataset-9e5d41","errorCode":null,"errorMessage":"dataset_streaming is HF-only; remove local_datasets / S3 source. Streaming is not supported with S3 datasets.","messagePattern":"dataset_streaming is HF-only; remove local_datasets / S3 source\\. Streaming is not supported with S3 datasets\\.","errorType":"http","errorClass":"HTTPException","httpStatus":400,"severity":"error","filePath":"studio/backend/routes/training.py","lineNumber":1374,"sourceCode":"            if request.eval_steps > 0:\n                train_split = request.train_split or \"train\"\n                if not request.eval_split or request.eval_split == train_split:\n                    raise HTTPException(\n                        status_code = 422,\n                        detail = \"dataset_streaming with evaluation requires a separate eval_split.\",\n                    )\n            # Streaming is HF-only: reject when the request also carries a local dataset path or an\n            # S3 config, since those sources cannot be streamed via HF's loader.\n            if request.local_datasets:\n                raise HTTPException(\n                    status_code = 400,\n                    detail = (\n                        \"dataset_streaming is HF-only; remove local_datasets / S3 source. \"\n                        \"Streaming is not supported with local file paths.\"\n                    ),\n                )\n            if request.s3_config is not None:\n                raise HTTPException(\n                    status_code = 400,\n                    detail = (\n                        \"dataset_streaming is HF-only; remove local_datasets / S3 source. \"\n                        \"Streaming is not supported with S3 datasets.\"\n                    ),\n                )\n            if request.dataset_known_cached or request.dataset_local_path:\n                raise HTTPException(\n                    status_code = 422,\n                    detail = (\n                        \"dataset_streaming streams from the Hub and cannot use the local \"\n                        \"dataset cache; disable streaming to train from the cached copy.\"\n                    ),\n                )\n        model_preflight = await asyncio.to_thread(\n            _reject_untrainable_model_request,\n            request,\n            resume_actual_model_repo_id,","sourceCodeStart":1356,"sourceCodeEnd":1392,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/routes/training.py#L1356-L1392","documentation":"HTTP 400 when dataset_streaming=true and s3_config is set: streaming is HF-Hub-only, and datasets staged from S3 are downloaded files that the HF streaming loader cannot consume. S3 sources and streaming are mutually exclusive.","triggerScenarios":"POST /training/start with dataset_streaming: true and s3_config != null.","commonSituations":"Switching a pipeline from S3-hosted data to a Hub dataset but leaving s3_config in the request payload; client SDK that always serializes an (empty) s3_config object.","solutions":["Set s3_config to null/omit it and provide hf_dataset when streaming","Or disable dataset_streaming to keep training from the S3 dataset","Ensure your client does not send an empty s3_config object - the field must be null, not {}"],"exampleFix":"// before\n{\"dataset_streaming\": true, \"s3_config\": {\"bucket\": \"data\", \"key\": \"ds.jsonl\"}}  // 400\n\n// after\n{\"dataset_streaming\": false, \"s3_config\": {\"bucket\": \"data\", \"key\": \"ds.jsonl\"}}\n// or\n{\"dataset_streaming\": true, \"hf_dataset\": \"org/ds\"}","handlingStrategy":"validation","validationCode":"def streaming_config_valid(p: dict) -> bool:\n    if not p.get(\"dataset_streaming\"):\n        return True\n    return p.get(\"s3_config\") is None","typeGuard":null,"tryCatchPattern":"resp = client.post(\"/training/start\", payload)\nif resp.status_code == 400 and \"S3\" in resp.text:\n    payload[\"s3_config\"] = None\n    payload[\"hf_dataset\"] = \"org/ds\"  # streaming needs a Hub dataset\n    resp = client.post(\"/training/start\", payload)","preventionTips":["Omit s3_config entirely (or send null) when dataset_streaming is true","Make sure your client SDK does not serialize an empty s3_config object by default","Pair streaming exclusively with an hf_dataset id"],"tags":["training","dataset-streaming","s3","http-400","mutually-exclusive"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}