hiyouga/LlamaFactory · error · ValueError

Cannot open {config_path} due to {str(err)}.

Error message

Cannot open {config_path} due to {str(err)}.

What it means

Raised while loading dataset configuration: dataset_info.json cannot be opened or parsed at the resolved path (local dataset_dir or a REMOTE: hub download). If dataset names were requested, the unreadable config is fatal; with no names requested it is tolerated (dataset_info = None).

Source

Thrown at src/llamafactory/data/parser.py:113

    if dataset_names is None:
        dataset_names = []

    if isinstance(dataset_dir, dict):
        dataset_info = dataset_dir
    elif dataset_dir == "ONLINE":
        dataset_info = None
    else:
        if dataset_dir.startswith("REMOTE:"):
            config_path = hf_hub_download(repo_id=dataset_dir[7:], filename=DATA_CONFIG, repo_type="dataset")
        else:
            config_path = os.path.join(dataset_dir, DATA_CONFIG)

        try:
            with open(config_path) as f:
                dataset_info = json.load(f)
        except Exception as err:
            if len(dataset_names) != 0:
                raise ValueError(f"Cannot open {config_path} due to {str(err)}.")

            dataset_info = None

    dataset_list: list[DatasetAttr] = []
    for name in dataset_names:
        if dataset_info is None:  # dataset_dir is ONLINE
            load_from = "ms_hub" if use_modelscope() else "om_hub" if use_openmind() else "hf_hub"
            dataset_attr = DatasetAttr(load_from, dataset_name=name)
            dataset_list.append(dataset_attr)
            continue

        if name not in dataset_info:
            raise ValueError(f"Undefined dataset {name} in {DATA_CONFIG}.")

        has_hf_url = "hf_hub_url" in dataset_info[name]
        has_ms_url = "ms_hub_url" in dataset_info[name]
        has_om_url = "om_hub_url" in dataset_info[name]

View on GitHub (pinned to f28afaf635)

Solutions

1. Check the path exists and is valid JSON: `python -c "import json;json.load(open('data/dataset_info.json'))"`.
2. Fix dataset_dir in the YAML to the folder that actually contains dataset_info.json.
3. For REMOTE:, verify repo id, HF_TOKEN, and connectivity (huggingface-cli download <repo> dataset_info.json --repo-type dataset).

Example fix

### before
dataset_dir: data/my_dataset  # no dataset_info.json here
dataset: alpaca
### after
dataset_dir: data  # folder containing dataset_info.json with an 'alpaca' entry

import json, os

path = os.path.join(dataset_dir, 'dataset_info.json')
assert os.path.isfile(path), f'missing {path}'
json.load(open(path))  # raises on malformed JSON before training starts

try:
    json.load(open(config_path))
except (OSError, json.JSONDecodeError) as e:
    raise SystemExit(f'dataset config unreadable: {config_path}: {e}') from e

When it happens

Trigger: Running train/api with dataset_dir pointing at a directory lacking dataset_info.json, a malformed JSON file, or REMOTE:repo where hf_hub_download fails (auth, network, missing file) while dataset_names is non-empty.

Common situations: Wrong dataset_dir path in YAML; JSON with trailing commas / comments; a dataset_dir set to the data file's folder instead of the config folder; hub outage or gated repo when using REMOTE:.

Related errors

• Undefined dataset {name} in {DATA_CONFIG}.
• Invalid JSON format in function message: {str([content])}.
• Invalid JSON format in tool description: {str([content])}.
• Unknown identifier: {node.id}
• tool_call value is not valid JSON: {content['value']!r}

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/a64e9b53eac202f1. Report an issue: GitHub.