geekcomputers/Python · critical · FileNotFoundError

Dataset directory not found: {split_dir}

Error message

Dataset directory not found: {split_dir}

What it means

Raised by ImageDataset (neuralforge) when the split directory (root/<split>, e.g. data/train) does not exist on disk. The loader builds the path from the root and split arguments and immediately checks os.path.exists, failing before any class discovery. It signals a data-location or download problem, not corrupted data.

Source

Thrown at ML/src/python/neuralforge/data/dataset.py:30

        root: str,
        transform: Optional[Callable] = None,
        target_transform: Optional[Callable] = None,
        split: str = 'train'
    ):
        self.root = root
        self.transform = transform
        self.target_transform = target_transform
        self.split = split
        
        self.samples = []
        self.class_to_idx = {}
        self._load_dataset()
    
    def _load_dataset(self):
        split_dir = os.path.join(self.root, self.split)
        
        if not os.path.exists(split_dir):
            raise FileNotFoundError(f"Dataset directory not found: {split_dir}")
        
        classes = sorted([d for d in os.listdir(split_dir) 
                         if os.path.isdir(os.path.join(split_dir, d))])
        
        self.class_to_idx = {cls_name: idx for idx, cls_name in enumerate(classes)}
        
        for class_name in classes:
            class_dir = os.path.join(split_dir, class_name)
            class_idx = self.class_to_idx[class_name]
            
            for img_name in os.listdir(class_dir):
                if img_name.lower().endswith(('.png', '.jpg', '.jpeg', '.bmp', '.gif')):
                    img_path = os.path.join(class_dir, img_name)
                    self.samples.append((img_path, class_idx))
    
    def __len__(self) -> int:
        return len(self.samples)
    

View on GitHub (pinned to 40f4cd2652)

Solutions

1. Verify the directory exists: ls root/<split> and confirm class subdirectories
2. Fix the root or split argument to point at the actual dataset layout
3. Download/prepare the dataset first (or pass download=True if supported)
4. If your layout is flat, restructure into root/train/<class>/*/jpg or wrap with a loader that matches your layout

Example fix

# before
ds = ImageNetDataset(root='./data/imagenet', split='train')  # dir missing

# after
import os
split_dir = os.path.join('./data/imagenet', 'train')
if not os.path.isdir(split_dir):
    download_or_prepare(split_dir)
ds = ImageNetDataset(root='./data/imagenet', split='train')

import os
split_dir = os.path.join(root, split)
if not os.path.isdir(split_dir):
    raise FileNotFoundError(f'Prepare {split_dir} before training')

def dataset_ready(root: str, split: str) -> bool:
    d = os.path.join(root, split)
    return os.path.isdir(d) and any(os.path.isdir(os.path.join(d, x)) for x in os.listdir(d))

try:
    ds = ImageNetDataset(root=root, split=split)
except FileNotFoundError as e:
    logger.error('Dataset missing: %s', e)
    sys.exit(2)  # or trigger download/preparation step

When it happens

Trigger: Passing a wrong root path; using a split name that does not exist as a subdirectory ('valid' when only train/test exist); expecting download=True to fetch data but the directory was never created.

Common situations: Running training on a machine where the dataset was never copied/downloaded; typos in root ('./dat'); directory-layout mismatch (images flat instead of train/<class>/); download flag not actually implemented for this dataset class.

Related errors

• Unknown dataset: {name}
• Invalid learning rate: {lr}
• Invalid epsilon value: {eps}

AI-assisted analysis of geekcomputers/Python@40f4cd2652 (2026-08-27). Data as JSON: /api/errors/15dbc0347c3767c1. Report an issue: GitHub.