{"record":{"id":"62006a3e22beca4f","repo":"hiyouga/LlamaFactory","slug":"unsupported-protocol-in-path-path-use-s3","errorCode":null,"errorMessage":"Unsupported protocol in path: {path}. Use 's3://' or 'gs://'.","messagePattern":"Unsupported protocol in path: (.+?)\\. Use 's3://' or 'gs://'\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/data/data_utils.py","lineNumber":166,"sourceCode":"\n            if len(eval_dataset):\n                dataset_module[\"eval_dataset\"] = eval_dataset\n\n    else:  # single dataset\n        dataset_module[\"train_dataset\"] = dataset\n\n    return dataset_module\n\n\ndef setup_fs(path: str, anon: bool = False) -> \"fsspec.AbstractFileSystem\":\n    r\"\"\"Set up a filesystem object based on the path protocol.\"\"\"\n    storage_options = {\"anon\": anon} if anon else {}\n    if path.startswith(\"s3://\"):\n        fs = fsspec.filesystem(\"s3\", **storage_options)\n    elif path.startswith((\"gs://\", \"gcs://\")):\n        fs = fsspec.filesystem(\"gcs\", **storage_options)\n    else:\n        raise ValueError(f\"Unsupported protocol in path: {path}. Use 's3://' or 'gs://'.\")\n\n    if not fs.exists(path):\n        raise ValueError(f\"Path does not exist: {path}.\")\n\n    return fs\n\n\ndef _read_json_with_fs(fs: \"fsspec.AbstractFileSystem\", path: str) -> list[Any]:\n    r\"\"\"Helper function to read JSON/JSONL files using fsspec.\"\"\"\n    with fs.open(path, \"r\") as f:\n        if path.endswith(\".jsonl\"):\n            return [json.loads(line) for line in f if line.strip()]\n        else:\n            return json.load(f)\n\n\ndef read_cloud_json(cloud_path: str) -> list[Any]:\n    r\"\"\"Read a JSON/JSONL file from cloud storage (S3 or GCS).","sourceCodeStart":148,"sourceCodeEnd":184,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/data/data_utils.py#L148-L184","documentation":"setup_fs only accepts s3:// and gs:// (or gcs://) path prefixes and raises ValueError for anything else. It is the entry point for cloud dataset loading, constructing an fsspec filesystem with optional anon=True for anonymous access.","triggerScenarios":"Calling the cloud JSON loading path (load云 datasets from S3/GCS) with an hdfs://, file://, azure://, or plain local path; missing or mistyped scheme (e.g. 's3:/bucket/x' with one slash).","commonSituations":"Pointing dataset paths at Azure Blob or local NFS paths while the data-loading code only supports S3/GCS; scheme typos in dataset_info.json entries.","solutions":["Use a fully-qualified s3://bucket/key or gs://bucket/key (gcs:// also accepted) path.","For local files, use ordinary local paths through the normal dataset loading route, not the cloud helper.","For other object stores, download/sync the data first (aws s3 cp, gsutil) or extend setup_fs with the needed fsspec protocol."],"exampleFix":"# before\n{\"file_name\": \"azure://container/data.json\"}\n\n# after\n# sync to local first, then\n{\"file_name\": \"data/local_copy/data.json\"}","handlingStrategy":"validation","validationCode":"def is_supported_cloud_path(p: str) -> bool:\n    return p.startswith((\"s3://\", \"gs://\", \"gcs://\"))\n\nassert is_supported_cloud_path(cloud_path), \"only s3:// and gs:// are supported\"","typeGuard":"def is_supported_cloud_path(p: str) -> bool:\n    return p.startswith((\"s3://\", \"gs://\", \"gcs://\"))","tryCatchPattern":null,"preventionTips":["Use fully-qualified store prefixes; avoid other schemes with this loader.","Local/other stores: sync data locally and use the normal dataset path."],"tags":["cloud-storage","fsspec","dataset","config"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}