{"record":{"id":"635264aed57db69d","repo":"hiyouga/LlamaFactory","slug":"path-does-not-exist-path","errorCode":null,"errorMessage":"Path does not exist: {path}.","messagePattern":"Path does not exist: (.+?)\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/data/data_utils.py","lineNumber":169,"sourceCode":"\n    else:  # single dataset\n        dataset_module[\"train_dataset\"] = dataset\n\n    return dataset_module\n\n\ndef setup_fs(path: str, anon: bool = False) -> \"fsspec.AbstractFileSystem\":\n    r\"\"\"Set up a filesystem object based on the path protocol.\"\"\"\n    storage_options = {\"anon\": anon} if anon else {}\n    if path.startswith(\"s3://\"):\n        fs = fsspec.filesystem(\"s3\", **storage_options)\n    elif path.startswith((\"gs://\", \"gcs://\")):\n        fs = fsspec.filesystem(\"gcs\", **storage_options)\n    else:\n        raise ValueError(f\"Unsupported protocol in path: {path}. Use 's3://' or 'gs://'.\")\n\n    if not fs.exists(path):\n        raise ValueError(f\"Path does not exist: {path}.\")\n\n    return fs\n\n\ndef _read_json_with_fs(fs: \"fsspec.AbstractFileSystem\", path: str) -> list[Any]:\n    r\"\"\"Helper function to read JSON/JSONL files using fsspec.\"\"\"\n    with fs.open(path, \"r\") as f:\n        if path.endswith(\".jsonl\"):\n            return [json.loads(line) for line in f if line.strip()]\n        else:\n            return json.load(f)\n\n\ndef read_cloud_json(cloud_path: str) -> list[Any]:\n    r\"\"\"Read a JSON/JSONL file from cloud storage (S3 or GCS).\n\n    Args:\n        cloud_path: str","sourceCodeStart":151,"sourceCodeEnd":187,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/data/data_utils.py#L151-L187","documentation":"After building the fsspec filesystem in setup_fs, fs.exists(path) is checked and ValueError is raised when the bucket/prefix does not exist. This distinguishes 'wrong credentials' from 'wrong path': existence probing happens with the constructed (anonymous or credentialed) filesystem.","triggerScenarios":"An s3:// or gs:// path with a typo'd bucket name, a wrong region endpoint, or an object the credentials cannot list; also anonymous access (anon=True tried first) against a private bucket whose existence is not visible anonymously — the credential retry may then raise here or an access error earlier.","commonSituations":"Private buckets accessed without credentials in the environment (no AWS_* / GOOGLE_APPLICATION_CREDENTIALS); bucket in another account; path uses the wrong separator or extra slash.","solutions":["Verify the path with the CLI of the store: aws s3 ls s3://bucket/prefix or gsutil ls gs://bucket/prefix.","For private data, export proper credentials (AWS credentials env vars or gcloud auth application-default login) before running.","Check bucket name spelling and region; ensure the listing permission exists (s3:ListBucket) since exists() needs it.","For public buckets, keep anon semantics in mind: confirm the objects really are public."],"exampleFix":"# shell: verify before running training\naws s3 ls s3://my-bucket/datasets/data.json  # must succeed\n\ngcloud auth application-default login  # for GCS private data","handlingStrategy":"validation","validationCode":"fs = fsspec.filesystem(\"s3\")  # or \"gcs\"\nif not fs.exists(cloud_path):\n    raise ValueError(f\"path not visible — check spelling, region, and credentials: {cloud_path}\")","typeGuard":null,"tryCatchPattern":"try:\n    fs = setup_fs(cloud_path, anon=True)\nexcept ValueError:\n    fs = setup_fs(cloud_path)  # credentialed retry mirrors the library's own fallback","preventionTips":["Verify paths with aws s3 ls / gsutil ls in the same shell env first.","Export credentials before runs for private buckets.","Ensure list permission exists — exists() probes via listing."],"tags":["cloud-storage","fsspec","dataset","permissions"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}