Comfy-Org/ComfyUI · error · ValueError

No shard files found in {dataset_dir}

Error message

No shard files found in {dataset_dir}

What it means

Thrown by the dataset-load node when the resolved dataset directory contains no files matching shard_*.pkl. The directory is derived from folder_name via get_dataset_dir, so the error means either nothing was ever saved there or the save used a different folder name/naming scheme.

Source

Thrown at comfy_extras/nodes_dataset.py:2081

            ],
        )

    @classmethod
    def execute(cls, folder_name):
        # Get dataset directory (searched across all dataset roots, traversal-safe)
        dataset_dir = get_dataset_dir(folder_name)

        # Find all shard files
        shard_files = sorted(
            [
                f
                for f in os.listdir(dataset_dir)
                if f.startswith("shard_") and f.endswith(".pkl")
            ]
        )

        if not shard_files:
            raise ValueError(f"No shard files found in {dataset_dir}")

        logging.info(f"Loading {len(shard_files)} shards from {dataset_dir}...")

        # Load all shards
        all_latents = []  # list[{"samples": tensor}]
        all_conditioning = []  # list[list[cond]]

        for shard_file in shard_files:
            shard_path = os.path.join(dataset_dir, shard_file)

            with open(shard_path, "rb") as f:
                shard_data = torch.load(f, weights_only=True)

            all_latents.extend(shard_data["latents"])
            all_conditioning.extend(shard_data["conditioning"])

            logging.info(f"Loaded {shard_file}: {len(shard_data['latents'])} samples")

View on GitHub (pinned to 1c6d8d45b3)

Solutions

1. Run the shard-save node first with the exact same folder_name, then load.
2. List the dataset directory manually and confirm files are named shard_XXXX... .pkl.
3. Check folder_name spelling/case — it is resolved via get_dataset_dir, so it must match the save-side value exactly.

dataset_dir = get_dataset_dir(folder_name)
shards = sorted(f for f in os.listdir(dataset_dir) if f.startswith('shard_') and f.endswith('.pkl'))
if not shards:
    raise FileNotFoundError(f'no shards in {dataset_dir}; run the shard-save node first')

try:
    dataset = load_dataset_node(...)
except ValueError as e:
    if 'No shard files found' in str(e):
        # run the save stage, then retry once
        save_dataset_node(...)
        dataset = load_dataset_node(...)
    else:
        raise

When it happens

Trigger: Calling the dataset loader with folder_name that was never written by the shard-save node; reading a directory where shards were deleted, still being written, or named differently (e.g. .pt/.safetensors instead of .pkl); folder_name case mismatch.

Common situations: Load-before-save ordering in a training workflow; typos or renamed folder between the save and load nodes; datasets copied to another machine with a different folder layout; trailing whitespace in folder_name.

Related errors

• No valid images found in input
• Invalid folder name {folder_name!r}: resolves outside of {ba
• folder_name must name a subfolder of the datasets directory,
• Dataset folder {folder_name!r} not found in: {', '.join(root
• No video files found in {sub_input_dir}

AI-assisted analysis of Comfy-Org/ComfyUI@1c6d8d45b3 (2026-08-14). Data as JSON: /api/errors/843a4e7a1a16339a. Report an issue: GitHub.