{"record":{"id":"d3e7c0a3ffe38c06","repo":"hiyouga/LlamaFactory","slug":"can-not-load-dataset-from-filepath","errorCode":null,"errorMessage":"Can not load dataset from {filepath}.","messagePattern":"Can not load dataset from (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/v1/plugins/data_plugins/loader.py","lineNumber":61,"sourceCode":"        Literal[\"arrow\", \"csv\", \"json\", \"parquet\", \"text\"]: Dataset builder name.\n    \"\"\"\n    filetype = os.path.splitext(path)[-1][1:]\n    if filetype in [\"arrow\", \"csv\", \"json\", \"jsonl\", \"parquet\", \"txt\"]:\n        return filetype.replace(\"jsonl\", \"json\").replace(\"txt\", \"text\")\n    else:\n        raise ValueError(f\"Unknown dataset filetype: {filetype}.\")\n\n\n@DataLoaderPlugin(\"local\").register()\ndef load_data_from_file(filepath: str, split: str, streaming: bool) -> HFDataset:\n    if os.path.isdir(filepath):\n        filetype = _get_builder_name(os.listdir(filepath)[0])\n        dataset = load_dataset(filetype, data_dir=filepath, split=split)\n    elif os.path.isfile(filepath):\n        filetype = _get_builder_name(filepath)\n        dataset = load_dataset(filetype, data_files=filepath, split=split)\n    else:\n        raise ValueError(f\"Can not load dataset from {filepath}.\")\n\n    if streaming:  # faster when data is streamed from local files\n        dataset = dataset.to_iterable_dataset()\n\n    return dataset\n\n\ndef adjust_data_index(\n    data_index: list[tuple[str, int]], size: int | None, weight: float | None\n) -> list[tuple[str, int]]:\n    \"\"\"Adjust dataset index by size and weight.\n\n    Args:\n        data_index (list[tuple[str, int]]): List of (dataset_name, sample_index).\n        size (Optional[int]): Desired dataset size.\n        weight (Optional[float]): Desired dataset weight.\n\n    Returns:","sourceCodeStart":43,"sourceCodeEnd":79,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/v1/plugins/data_plugins/loader.py#L43-L79","documentation":"The local data loader only handles two cases: an existing directory or an existing file. If the path is neither (nonexistent, a glob pattern, or a URL), it raises this ValueError. Note _get_builder_name may raise first for directories whose first file has a bad extension.","triggerScenarios":"Passing a nonexistent path in dataset_dir; passing a glob like 'data/*.json'; passing an HF hub repo id (e.g. 'alpaca') where the local loader plugin is selected instead of the hub loader.","commonSituations":"Typo in the dataset path; using a relative path while the process cwd differs; expecting local-loader semantics for hub datasets because the dataset_info entry was misconfigured.","solutions":["Check the path exists: correct typos, use absolute paths, or fix the working directory.","Expand globs yourself and list concrete files in the config.","For HuggingFace hub datasets, register them properly (dataset_info.json or the hub loader) instead of the local file loader.","Verify with `ls <path>` from the same shell that launches training."],"exampleFix":"# before\ndataset_dir: ./datset/train.json   # typo\n\n# after\ndataset_dir: ./dataset/train.json  # verified with ls","handlingStrategy":"validation","validationCode":"import os\nassert os.path.isdir(dataset_dir) or os.path.isfile(dataset_dir), f'path not found: {dataset_dir}'","typeGuard":"def is_loadable_path(p: str) -> bool:\n    \"\"\"True when p is an existing file or directory.\"\"\"\n    return os.path.exists(p)","tryCatchPattern":"try:\n    ds = load_data_from_file(p, split, streaming)\nexcept ValueError as e:\n    if 'Can not load dataset' in str(e):\n        raise SystemExit(f'check path: {p!r} (cwd={os.getcwd()})') from None\n    raise","preventionTips":["Use absolute paths in training configs.","Add a `ls` sanity step in launch scripts.","Expand globs explicitly in data prep, never pass them through."],"tags":["data","path","dataset-loading","config"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}