huggingface/pytorch-image-models · warning

Error processing sample index {idx}. Error: {e}. Skipping sa

Error message

Error processing sample index {idx}. Error: {e}. Skipping sample.

What it means

During NaFlexDataset iteration, per-sample loading and transform execution is wrapped in try/except; any non-IndexError exception (corrupt image, decode failure, transform error) triggers a warning and the sample is skipped so training continues.

Source

Thrown at timm/data/naflex_dataset.py:555

                    # Get original image and label from map-style dataset
                    img, label = self.base_dataset[idx]

                    # Apply transform if available
                    # Handle cases where transform might return None or fail
                    processed_img = transform(img) if transform else img
                    if processed_img is None:
                        warnings.warn(f"Transform returned None for index {idx}. Skipping sample.")
                        continue

                    batch_imgs.append(processed_img)
                    batch_targets.append(label)

                except IndexError:
                     warnings.warn(f"IndexError encountered for index {idx} (possibly due to padding/repeated indices). Skipping sample.")
                     continue
                except Exception as e:
                    # Log other potential errors during data loading/processing
                    warnings.warn(f"Error processing sample index {idx}. Error: {e}. Skipping sample.")
                    continue # Skip problematic sample

            if self.mixup_fn is not None:
                batch_imgs, batch_targets = self.mixup_fn(batch_imgs, batch_targets)

            batch_imgs = [batch_patchifier(img) for img in batch_imgs]
            batch_samples = list(zip(batch_imgs, batch_targets))
            if batch_samples: # Only yield if we successfully processed samples
                # Collate the processed samples into a batch
                yield self.collate_fns[seq_len](batch_samples)

            # If batch_samples is empty after processing 'indices', an empty batch is skipped.

View on GitHub (pinned to 9a5261e31b)

Solutions

1. Inspect the warned indices and remove/repair those dataset entries (filter the underlying dataset)
2. Verify files exist and open with PIL before training; normalize images to RGB in the transform
3. If warnings are frequent, run a preprocessing pass to validate/clean the dataset

Example fix

# before
ds = NaFlexDataset(...)
# after — pre-filter broken samples
from PIL import Image
clean = [i for i in range(len(base_ds)) if _opens_ok(base_ds, i)]
base_ds = torch.utils.data.Subset(base_ds, clean)
ds = NaFlexDataset(base_ds, ...)

from PIL import Image\ndef ok(i, ds):\n    try:\n        img, _ = ds[i]; Image.open if False else None\n        return img is not None\n    except Exception:\n        return False

When it happens

Trigger: A dataset item whose image is corrupt/unreadable, or a transform (e.g. naflex patchify/resize) raising on an odd-sized or grayscale image; PIL decode errors; FileNotFoundError for missing files.

Common situations: Web-scraped datasets with broken files; mixed image modes (L/CMYK) hitting transforms expecting RGB; filesystem issues. Repeated warnings indicate real data corruption, not a bug in the loader.

Related errors

• Transform returned None for index {idx}. Skipping sample.
• Invalid class map file, expected a dict ({class_map_path}).
• Dataset length is unknown, please pass `num_samples` explici
• Found 0 images in subfolders of {root}. Supported image exte
• Calculated batch size <= 0 (seq_len={seq_len}, remaining={re

AI-assisted analysis of huggingface/pytorch-image-models@9a5261e31b (2026-08-27). Data as JSON: /api/errors/1ee86fb9c467e488. Report an issue: GitHub.