xinntao/Real-ESRGAN · warning

File client error: {e}, remaining retry times: {retry - 1}

Error message

File client error: {e}, remaining retry times: {retry - 1}

What it means

This is not a raised exception but a logged warning from RealESRGAN's __getitem__ retry loop: the file client (disk, lmdb, or network backend) raised IOError/OSError while reading the GT image at gt_path. The dataset retries up to 3 times, each time picking a random different index and sleeping 1s, to ride out transient I/O problems or server congestion. If all retries fail the loop exits and the (possibly stale) img_bytes is used, typically causing a downstream error in the image decoder.

Source

Thrown at realesrgan/data/realesrgan_dataset.py:97

        # TODO: kernel range is now hard-coded, should be in the configure file
        self.pulse_tensor = torch.zeros(21, 21).float()  # convolving with pulse tensor brings no blurry effect
        self.pulse_tensor[10, 10] = 1

    def __getitem__(self, index):
        if self.file_client is None:
            self.file_client = FileClient(self.io_backend_opt.pop('type'), **self.io_backend_opt)

        # -------------------------------- Load gt images -------------------------------- #
        # Shape: (h, w, c); channel order: BGR; image range: [0, 1], float32.
        gt_path = self.paths[index]
        # avoid errors caused by high latency in reading files
        retry = 3
        while retry > 0:
            try:
                img_bytes = self.file_client.get(gt_path, 'gt')
            except (IOError, OSError) as e:
                logger = get_root_logger()
                logger.warn(f'File client error: {e}, remaining retry times: {retry - 1}')
                # change another file to read
                index = random.randint(0, self.__len__())
                gt_path = self.paths[index]
                time.sleep(1)  # sleep 1s for occasional server congestion
            else:
                break
            finally:
                retry -= 1
        img_gt = imfrombytes(img_bytes, float32=True)

        # -------------------- Do augmentation for training: flip, rotation -------------------- #
        img_gt = augment(img_gt, self.opt['use_hflip'], self.opt['use_rot'])

        # crop or pad to 400
        # TODO: 400 is hard-coded. You may change it accordingly
        h, w = img_gt.shape[0:2]
        crop_pad_size = 400
        # pad

View on GitHub (pinned to a4abfb2979)

Solutions

1. Verify every path in meta_info.txt exists and is non-empty: check for missing/zero-byte files and regenerate meta_info.txt against the current dataset directory
2. If the file is truncated/corrupt, re-download or restore it (or rebuild the LMDB with create_lmdb.py) — a truncated image otherwise surfaces later as a decode error
3. For network/LMDB backends, check mount health / permissions (ls -la, stat the failing path) and that data.mdb/lock.mdb are readable by the training user
4. If caused by transient NFS congestion, no action needed: the built-in 3-retry + 1s-sleep loop usually recovers; persisting warnings indicate real corruption

Example fix

# before: meta_info.txt references deleted frames
0001.png
0002.png   # deleted on disk

# after: regenerate meta info from existing files
import os
with open('meta_info.txt', 'w') as f:
    for name in sorted(os.listdir('gt_dir')):
        if name.endswith('.png'):
            f.write(name + '\n')

import os
missing = [p for p in dataset.paths if not os.path.exists(os.path.join(dataset.gt_folder, p + '.png'))]
if missing:
    raise RuntimeError(f'{len(missing)} GT files missing, e.g. {missing[:5]}')

try:
    lq, gt = dataset[i]
except Exception as e:  # exhausted retries surface as decode/None errors
    logger.warning(f'sample {i} bad, skipping: {e}')
    i = random.randint(0, len(dataset) - 1)
    lq, gt = dataset[i]

When it happens

Trigger: Calling dataset[i] / DataLoader iteration when a GT file listed in meta_info.txt is missing, truncated, or unreadable (disk backend); a corrupted or improperly closed LMDB (lmdb backend); or an NFS/network mount that intermittently drops (IOError/OSError from open/read). Files deleted or moved after meta_info.txt was generated also produce it.

Common situations: meta_info.txt generated before some images were deleted/moved; partially downloaded datasets with zero-byte or truncated PNG/JPG files; LMDB built on a machine that crashed mid-creation; training on NFS with flaky connectivity; file permission errors after copying a dataset between users/containers.

Related errors

• 'dataroot_gt' should end with '.lmdb', but received {self.gt

AI-assisted analysis of xinntao/Real-ESRGAN@a4abfb2979 (2026-08-27). Data as JSON: /api/errors/934d40e401661a30. Report an issue: GitHub.