{"record":{"id":"4b1bb47eee3b0341","repo":"huggingface/pytorch-image-models","slug":"invalid-split-definition-num-samples-not-specifie","errorCode":null,"errorMessage":"Invalid split definition, num_samples not specified in train mode.","messagePattern":"Invalid split definition, num_samples not specified in train mode\\.","errorType":"error_code","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"timm/data/readers/reader_wds.py","lineNumber":308,"sourceCode":"        self.common_seed = seed  # a seed that's fixed across all worker / distributed instances\n        self.shard_shuffle_size = 500\n        self.sample_shuffle_size = sample_shuffle_size or SAMPLE_SHUFFLE_SIZE\n        self.sample_initial_size = sample_initial_size or SAMPLE_INITIAL_SIZE\n\n        self.input_key = input_key\n        self.input_img_mode = input_img_mode\n        self.target_key = target_key\n        self.filename_key = filename_key\n        self.key_ext = '.JPEG'  # extension to add to key for original filenames (DS specific, default ImageNet)\n\n        self.info = _load_info(self.root)\n        self.split_info = _parse_split_info(split, self.info)\n        if num_samples is not None:\n            self.num_samples = num_samples\n        else:\n            self.num_samples = self.split_info.num_samples\n        if is_training and not self.num_samples:\n            raise RuntimeError(f'Invalid split definition, num_samples not specified in train mode.')\n        self.remap_class = False\n        if class_map:\n            self.class_to_idx = load_class_map(class_map)\n            self.remap_class = True\n        else:\n            self.class_to_idx = {}\n\n        # Distributed world state\n        self.dist_rank = 0\n        self.dist_num_replicas = 1\n        if dist.is_available() and dist.is_initialized() and dist.get_world_size() > 1:\n            self.dist_rank = dist.get_rank()\n            self.dist_num_replicas = dist.get_world_size()\n\n        # Attributes that are updated in _lazy_init\n        self.worker_info = None\n        self.worker_id = 0\n        self.worker_seed = seed  # seed unique to each worker instance","sourceCodeStart":290,"sourceCodeEnd":326,"githubUrl":"https://github.com/huggingface/pytorch-image-models/blob/9a5261e31b3b5128526eb2658333b4c0a54464ae/timm/data/readers/reader_wds.py#L290-L326","documentation":"In training mode the WebDataset reader must know the dataset length (for the LR scheduler and epoch semantics); neither num_samples nor the split's num_samples in info.json provided a positive value, so construction aborts.","triggerScenarios":"ReaderWds(..., is_training=True) where num_samples is None and info.json's split entry has num_samples missing, 0, or null.","commonSituations":"Hand-written or converted info.json that omits num_samples; using a validation-only shard layout for training; a split-name mismatch resolved to an empty split entry.","solutions":["Pass num_samples explicitly to the reader / train script.","Add \"num_samples\": <int> to the split entry in the dataset's info.json.","Regenerate info.json with the webdataset build tooling that records sample counts."],"exampleFix":"# info.json (before): {\"splits\": {\"train\": {\"filenames\": [...]}}\n# after:\n{\"splits\": {\"train\": {\"num_samples\": 1281167, \"filenames\": [...]}}}\n# or in code:\nreader = ReaderWds(root, is_training=True, num_samples=1_281_167)","handlingStrategy":"validation","validationCode":"assert num_samples or (info.get('splits',{}).get(split,{}) or {}).get('num_samples'), 'pass num_samples'\nreader = ReaderWds(root, split=split, is_training=True, num_samples=num_samples)","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Always record num_samples per split when authoring info.json.","Pass explicit num_samples in training scripts.","Validate info.json schema at dataset build time."],"tags":["timm","webdataset","num-samples","training"],"backgroundTag":"unknown-dataset-length","analyzedSha":"9a5261e31b3b5128526eb2658333b4c0a54464ae","analyzedAt":"2026-08-27T02:34:25.417Z","schemaVersion":2},"datasetVersion":"2026-08-27T03:17:27.898Z"}