{"record":{"id":"68090c0b67d4f21e","repo":"FoundationAgents/MetaGPT","slug":"invalid-split-split-for-dataset-dataset-name-or","errorCode":null,"errorMessage":"Invalid split {split} for dataset {dataset_name_or_path}","messagePattern":"Invalid split (.+?) for dataset (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"metagpt/tools/swe_agent_commands/swe_agent_utils.py","lineNumber":27,"sourceCode":"    recording = False\n    for line in command_output.split(\"\\n\"):\n        if line.startswith(\"diff --git\"):\n            recording = True\n        if recording:\n            patch_lines.append(line)\n    return \"\\n\".join(patch_lines)\n\n\ndef load_hf_dataset(dataset_name_or_path: str, cache_dir, split: str = \"test\", existing_ids: list = []):\n    data_dir = cache_dir / dataset_name_or_path\n    if Path(data_dir).exists():\n        dataset = load_from_disk(data_dir)\n    else:\n        dataset = load_dataset(dataset_name_or_path)\n        dataset.save_to_disk(data_dir)\n    print(dataset)\n    if split not in dataset:\n        raise ValueError(f\"Invalid split {split} for dataset {dataset_name_or_path}\")\n    dataset = dataset[split]\n    np.array(list(map(len, dataset[\"instance_id\"])))\n\n    if existing_ids:\n        dataset = dataset.filter(\n            lambda x: x[\"instance_id\"] not in existing_ids,\n            desc=\"Filtering out existing ids\",\n            load_from_cache_file=False,\n        )\n\n    return dataset\n","sourceCodeStart":9,"sourceCodeEnd":39,"githubUrl":"https://github.com/FoundationAgents/MetaGPT/blob/11cdf466d042aece04fc6cfd13b28e1a70341b1f/metagpt/tools/swe_agent_commands/swe_agent_utils.py#L9-L39","documentation":"ValueError from load_hf_dataset in swe_agent_utils: the requested split name (default 'test') is not present in the loaded Hugging Face dataset (which may be a DatasetDict of train/validation/test, or a cached on-disk variant with different splits). The function caches datasets under cache_dir and reloads with load_from_disk, so a stale cache can also carry different splits than the remote dataset.","triggerScenarios":"load_hf_dataset('princeton-nlp/SWE-bench_Lite', cache_dir, split='validation') when the dataset only has 'test' and 'dev'; a cached copy on disk saved with different/renamed splits.","commonSituations":"Dataset revisions renaming splits (e.g. 'validation' -> 'dev'); code written against SWE-bench full reused for Lite/Verified variants; stale on-disk cache in cache_dir predating a split rename.","solutions":["Print/inspect the available splits (the function already print(dataset)) and use one of them, typically 'test'","If the cache is stale, delete cache_dir/<dataset_name> so it re-downloads fresh","Pass the correct split explicitly instead of relying on the 'test' default"],"exampleFix":"# before\nds = load_hf_dataset(\"princeton-nlp/SWE-bench_Lite\", cache_dir=cache, split=\"validation\")\n# after\nfrom datasets import load_dataset\nprint(load_dataset(\"princeton-nlp/SWE-bench_Lite\").keys())  # e.g. dict_keys(['test'])\nds = load_hf_dataset(\"princeton-nlp/SWE-bench_Lite\", cache_dir=cache, split=\"test\")","handlingStrategy":"validation","validationCode":"from datasets import load_from_disk, load_dataset\nfrom pathlib import Path\ncache = Path(cache_dir) / dataset_name_or_path\nds = load_from_disk(cache) if cache.exists() else load_dataset(dataset_name_or_path)\nif split not in ds:\n    raise SystemExit(f\"splits available: {list(ds.keys())}\")","typeGuard":"def has_split(dataset, split: str) -> bool:\n    return split in dataset  # works for DatasetDict","tryCatchPattern":"try:\n    ds = load_hf_dataset(name, cache_dir, split=split)\nexcept ValueError:\n    from datasets import load_dataset\n    split = next(iter(load_dataset(name).keys()))  # pick first available split\n    ds = load_hf_dataset(name, cache_dir, split=split)","preventionTips":["Check dataset.keys() before requesting a split","Delete stale cache_dir entries when upstream datasets rename splits","Pin dataset revisions for reproducible pipelines"],"tags":["datasets","huggingface","swe-bench","data"],"backgroundTag":null,"analyzedSha":"11cdf466d042aece04fc6cfd13b28e1a70341b1f","analyzedAt":"2026-08-14T23:20:02.994Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}