{"record":{"id":"ad64fdbc2c22a092","repo":"unslothai/unsloth","slug":"dataset-streaming-is-not-yet-supported-on-apple-si","errorCode":null,"errorMessage":"dataset_streaming is not yet supported on Apple Silicon (MLX); the MLX loader materializes the full dataset.","messagePattern":"dataset_streaming is not yet supported on Apple Silicon \\(MLX\\); the MLX loader materializes the full dataset\\.","errorType":"http","errorClass":"HTTPException","httpStatus":400,"severity":"error","filePath":"studio/backend/routes/training.py","lineNumber":1342,"sourceCode":"\n        if request.dataset_streaming:\n            if not request.hf_dataset:\n                raise HTTPException(\n                    status_code = 400,\n                    detail = \"dataset_streaming requires hf_dataset; streaming is not supported for local datasets.\",\n                )\n            if request.is_dataset_image or request.is_dataset_audio:\n                raise HTTPException(\n                    status_code = 400,\n                    detail = \"dataset_streaming is not supported for vision or audio datasets.\",\n                )\n            if request.is_embedding:\n                raise HTTPException(\n                    status_code = 400,\n                    detail = \"dataset_streaming is not supported for embedding training; the embedding loader needs the full dataset.\",\n                )\n            if _hw.DEVICE == _hw.DeviceType.MLX:\n                raise HTTPException(\n                    status_code = 400,\n                    detail = \"dataset_streaming is not yet supported on Apple Silicon (MLX); the MLX loader materializes the full dataset.\",\n                )\n            if request.max_steps is None or request.max_steps <= 0:\n                raise HTTPException(\n                    status_code = 422,\n                    detail = \"dataset_streaming requires max_steps > 0 because streaming datasets have no known length.\",\n                )\n            if request.train_on_completions:\n                raise HTTPException(\n                    status_code = 422,\n                    detail = \"dataset_streaming is not supported with train_on_completions yet.\",\n                )\n            if request.eval_steps > 0:\n                train_split = request.train_split or \"train\"\n                if not request.eval_split or request.eval_split == train_split:\n                    raise HTTPException(\n                        status_code = 422,","sourceCodeStart":1324,"sourceCodeEnd":1360,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/routes/training.py#L1324-L1360","documentation":"HTTP 400 when dataset_streaming=true and the detected hardware is MLX (Apple Silicon): the MLX dataset loader materializes the full dataset and has no streaming path, so the combination is rejected rather than silently ignored.","triggerScenarios":"POST /training/start with dataset_streaming: true on a Mac where utils.hardware reports DEVICE == DeviceType.MLX.","commonSituations":"Config developed on a CUDA/Linux box reused on an Apple Silicon machine; assuming the streaming flag is platform-independent.","solutions":["Disable dataset_streaming on Apple Silicon (MLX) hosts","Or run the streaming training on a CUDA host","Gate the streaming flag on platform in your client config so it is only set where supported"],"exampleFix":"// before (on a Mac)\n{\"dataset_streaming\": true, \"hf_dataset\": \"org/ds\"}  // 400\n\n// after\n{\"dataset_streaming\": false, \"hf_dataset\": \"org/ds\"}","handlingStrategy":"validation","validationCode":"import platform\n\ndef streaming_supported_here() -> bool:\n    return platform.machine() != \"arm64\" or not platform.system() == \"Darwin\"  # proxy for MLX hosts\n\ndef streaming_config_valid(p: dict) -> bool:\n    return (not p.get(\"dataset_streaming\")) or streaming_supported_here()","typeGuard":null,"tryCatchPattern":"resp = client.post(\"/training/start\", payload)\nif resp.status_code == 400 and \"MLX\" in resp.text:\n    payload[\"dataset_streaming\"] = False\n    resp = client.post(\"/training/start\", payload)","preventionTips":["Gate the streaming flag on host platform in shared configs","Do not port CUDA training configs verbatim to Apple Silicon","Remember MLX materializes the full dataset: size disk accordingly"],"tags":["training","dataset-streaming","mlx","apple-silicon","platform","http-400"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}