{"record":{"id":"df5434c2c594046f","repo":"huggingface/pytorch-image-models","slug":"indexerror-encountered-for-index-idx-possibly-d","errorCode":null,"errorMessage":"IndexError encountered for index {idx} (possibly due to padding/repeated indices). Skipping sample.","messagePattern":"IndexError encountered for index (.+?) \\(possibly due to padding/repeated indices\\)\\. Skipping sample\\.","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"timm/data/naflex_dataset.py","lineNumber":551,"sourceCode":"            batch_imgs = []\n            batch_targets = []\n            for idx in indices:\n                try:\n                    # Get original image and label from map-style dataset\n                    img, label = self.base_dataset[idx]\n\n                    # Apply transform if available\n                    # Handle cases where transform might return None or fail\n                    processed_img = transform(img) if transform else img\n                    if processed_img is None:\n                        warnings.warn(f\"Transform returned None for index {idx}. Skipping sample.\")\n                        continue\n\n                    batch_imgs.append(processed_img)\n                    batch_targets.append(label)\n\n                except IndexError:\n                     warnings.warn(f\"IndexError encountered for index {idx} (possibly due to padding/repeated indices). Skipping sample.\")\n                     continue\n                except Exception as e:\n                    # Log other potential errors during data loading/processing\n                    warnings.warn(f\"Error processing sample index {idx}. Error: {e}. Skipping sample.\")\n                    continue # Skip problematic sample\n\n            if self.mixup_fn is not None:\n                batch_imgs, batch_targets = self.mixup_fn(batch_imgs, batch_targets)\n\n            batch_imgs = [batch_patchifier(img) for img in batch_imgs]\n            batch_samples = list(zip(batch_imgs, batch_targets))\n            if batch_samples: # Only yield if we successfully processed samples\n                # Collate the processed samples into a batch\n                yield self.collate_fns[seq_len](batch_samples)\n\n            # If batch_samples is empty after processing 'indices', an empty batch is skipped.\n","sourceCodeStart":533,"sourceCodeEnd":568,"githubUrl":"https://github.com/huggingface/pytorch-image-models/blob/9a5261e31b3b5128526eb2658333b4c0a54464ae/timm/data/naflex_dataset.py#L533-L568","documentation":"While iterating, NaFlexDataset catches IndexError from base_dataset[idx] or the transform — typically when padding added indices beyond the real dataset length — warns and skips. It is the expected mechanism for tolerating padded/duplicate indices rather than crashing the loader.","triggerScenarios":"Distributed padding extends indices past len(base_dataset) (pad indices >= dataset size); an out-of-range idx computed from a mismatched schedule; a Subset/wrapper whose length shrank after caching indices.","commonSituations":"Distributed training with drop_last=False-style padding; mixing Subset views with indices computed from the parent dataset. Occasional skips are by design; many skips mean index computation is wrong.","solutions":["If only pad indices trigger it, it's expected behavior — no action needed","If frequent, verify how indices are generated vs len(base_dataset) (e.g. Subset ranges)","Ensure base_dataset length is stable between schedule creation and iteration"],"exampleFix":"# before\nidxs = list(range(len(parent_ds)))  # later wrapped in Subset(base, idxs) of smaller size\n# after\nsub = Subset(base, idxs)\nidxs = list(range(len(sub)))  # schedule from the actual visible length","handlingStrategy":"fallback","validationCode":"assert all(0 <= i < len(base_dataset) for i in batch_indices), 'index out of dataset range'","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Compute indices from len() of the actual dataset object being wrapped","Expect occasional pad-index skips in distributed mode"],"tags":["naflex","index-error","padding","timm"],"backgroundTag":"index-out-of-range","analyzedSha":"9a5261e31b3b5128526eb2658333b4c0a54464ae","analyzedAt":"2026-08-27T02:34:25.417Z","schemaVersion":2},"datasetVersion":"2026-08-27T03:17:27.898Z"}