{"record":{"id":"843a4e7a1a16339a","repo":"Comfy-Org/ComfyUI","slug":"no-shard-files-found-in-dataset-dir","errorCode":null,"errorMessage":"No shard files found in {dataset_dir}","messagePattern":"No shard files found in (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"comfy_extras/nodes_dataset.py","lineNumber":2081,"sourceCode":"            ],\n        )\n\n    @classmethod\n    def execute(cls, folder_name):\n        # Get dataset directory (searched across all dataset roots, traversal-safe)\n        dataset_dir = get_dataset_dir(folder_name)\n\n        # Find all shard files\n        shard_files = sorted(\n            [\n                f\n                for f in os.listdir(dataset_dir)\n                if f.startswith(\"shard_\") and f.endswith(\".pkl\")\n            ]\n        )\n\n        if not shard_files:\n            raise ValueError(f\"No shard files found in {dataset_dir}\")\n\n        logging.info(f\"Loading {len(shard_files)} shards from {dataset_dir}...\")\n\n        # Load all shards\n        all_latents = []  # list[{\"samples\": tensor}]\n        all_conditioning = []  # list[list[cond]]\n\n        for shard_file in shard_files:\n            shard_path = os.path.join(dataset_dir, shard_file)\n\n            with open(shard_path, \"rb\") as f:\n                shard_data = torch.load(f, weights_only=True)\n\n            all_latents.extend(shard_data[\"latents\"])\n            all_conditioning.extend(shard_data[\"conditioning\"])\n\n            logging.info(f\"Loaded {shard_file}: {len(shard_data['latents'])} samples\")\n","sourceCodeStart":2063,"sourceCodeEnd":2099,"githubUrl":"https://github.com/Comfy-Org/ComfyUI/blob/1c6d8d45b3693bfbb32385b410d813a7fd6be216/comfy_extras/nodes_dataset.py#L2063-L2099","documentation":"Thrown by the dataset-load node when the resolved dataset directory contains no files matching shard_*.pkl. The directory is derived from folder_name via get_dataset_dir, so the error means either nothing was ever saved there or the save used a different folder name/naming scheme.","triggerScenarios":"Calling the dataset loader with folder_name that was never written by the shard-save node; reading a directory where shards were deleted, still being written, or named differently (e.g. .pt/.safetensors instead of .pkl); folder_name case mismatch.","commonSituations":"Load-before-save ordering in a training workflow; typos or renamed folder between the save and load nodes; datasets copied to another machine with a different folder layout; trailing whitespace in folder_name.","solutions":["Run the shard-save node first with the exact same folder_name, then load.","List the dataset directory manually and confirm files are named shard_XXXX... .pkl.","Check folder_name spelling/case — it is resolved via get_dataset_dir, so it must match the save-side value exactly."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"dataset_dir = get_dataset_dir(folder_name)\nshards = sorted(f for f in os.listdir(dataset_dir) if f.startswith('shard_') and f.endswith('.pkl'))\nif not shards:\n    raise FileNotFoundError(f'no shards in {dataset_dir}; run the shard-save node first')","typeGuard":null,"tryCatchPattern":"try:\n    dataset = load_dataset_node(...)\nexcept ValueError as e:\n    if 'No shard files found' in str(e):\n        # run the save stage, then retry once\n        save_dataset_node(...)\n        dataset = load_dataset_node(...)\n    else:\n        raise","preventionTips":["Use the identical folder_name string in save and load nodes (watch case/whitespace).","Verify shard_*.pkl files exist on disk before queuing the training stage.","Treat load-before-save ordering as a workflow bug and fix the order."],"tags":["comfyui","dataset","file-not-found","training"],"backgroundTag":null,"analyzedSha":"1c6d8d45b3693bfbb32385b410d813a7fd6be216","analyzedAt":"2026-08-14T19:37:18.893Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}