{"record":{"id":"934d40e401661a30","repo":"xinntao/Real-ESRGAN","slug":"file-client-error-e-remaining-retry-times-re","errorCode":null,"errorMessage":"File client error: {e}, remaining retry times: {retry - 1}","messagePattern":"File client error: (.+?), remaining retry times: (.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"realesrgan/data/realesrgan_dataset.py","lineNumber":97,"sourceCode":"        # TODO: kernel range is now hard-coded, should be in the configure file\n        self.pulse_tensor = torch.zeros(21, 21).float()  # convolving with pulse tensor brings no blurry effect\n        self.pulse_tensor[10, 10] = 1\n\n    def __getitem__(self, index):\n        if self.file_client is None:\n            self.file_client = FileClient(self.io_backend_opt.pop('type'), **self.io_backend_opt)\n\n        # -------------------------------- Load gt images -------------------------------- #\n        # Shape: (h, w, c); channel order: BGR; image range: [0, 1], float32.\n        gt_path = self.paths[index]\n        # avoid errors caused by high latency in reading files\n        retry = 3\n        while retry > 0:\n            try:\n                img_bytes = self.file_client.get(gt_path, 'gt')\n            except (IOError, OSError) as e:\n                logger = get_root_logger()\n                logger.warn(f'File client error: {e}, remaining retry times: {retry - 1}')\n                # change another file to read\n                index = random.randint(0, self.__len__())\n                gt_path = self.paths[index]\n                time.sleep(1)  # sleep 1s for occasional server congestion\n            else:\n                break\n            finally:\n                retry -= 1\n        img_gt = imfrombytes(img_bytes, float32=True)\n\n        # -------------------- Do augmentation for training: flip, rotation -------------------- #\n        img_gt = augment(img_gt, self.opt['use_hflip'], self.opt['use_rot'])\n\n        # crop or pad to 400\n        # TODO: 400 is hard-coded. You may change it accordingly\n        h, w = img_gt.shape[0:2]\n        crop_pad_size = 400\n        # pad","sourceCodeStart":79,"sourceCodeEnd":115,"githubUrl":"https://github.com/xinntao/Real-ESRGAN/blob/a4abfb2979a7bbff3f69f58f58ae324608821e27/realesrgan/data/realesrgan_dataset.py#L79-L115","documentation":"This is not a raised exception but a logged warning from RealESRGAN's __getitem__ retry loop: the file client (disk, lmdb, or network backend) raised IOError/OSError while reading the GT image at gt_path. The dataset retries up to 3 times, each time picking a random different index and sleeping 1s, to ride out transient I/O problems or server congestion. If all retries fail the loop exits and the (possibly stale) img_bytes is used, typically causing a downstream error in the image decoder.","triggerScenarios":"Calling dataset[i] / DataLoader iteration when a GT file listed in meta_info.txt is missing, truncated, or unreadable (disk backend); a corrupted or improperly closed LMDB (lmdb backend); or an NFS/network mount that intermittently drops (IOError/OSError from open/read). Files deleted or moved after meta_info.txt was generated also produce it.","commonSituations":"meta_info.txt generated before some images were deleted/moved; partially downloaded datasets with zero-byte or truncated PNG/JPG files; LMDB built on a machine that crashed mid-creation; training on NFS with flaky connectivity; file permission errors after copying a dataset between users/containers.","solutions":["Verify every path in meta_info.txt exists and is non-empty: check for missing/zero-byte files and regenerate meta_info.txt against the current dataset directory","If the file is truncated/corrupt, re-download or restore it (or rebuild the LMDB with create_lmdb.py) — a truncated image otherwise surfaces later as a decode error","For network/LMDB backends, check mount health / permissions (ls -la, stat the failing path) and that data.mdb/lock.mdb are readable by the training user","If caused by transient NFS congestion, no action needed: the built-in 3-retry + 1s-sleep loop usually recovers; persisting warnings indicate real corruption"],"exampleFix":"# before: meta_info.txt references deleted frames\n0001.png\n0002.png   # deleted on disk\n\n# after: regenerate meta info from existing files\nimport os\nwith open('meta_info.txt', 'w') as f:\n    for name in sorted(os.listdir('gt_dir')):\n        if name.endswith('.png'):\n            f.write(name + '\\n')","handlingStrategy":"retry","validationCode":"import os\nmissing = [p for p in dataset.paths if not os.path.exists(os.path.join(dataset.gt_folder, p + '.png'))]\nif missing:\n    raise RuntimeError(f'{len(missing)} GT files missing, e.g. {missing[:5]}')","typeGuard":null,"tryCatchPattern":"try:\n    lq, gt = dataset[i]\nexcept Exception as e:  # exhausted retries surface as decode/None errors\n    logger.warning(f'sample {i} bad, skipping: {e}')\n    i = random.randint(0, len(dataset) - 1)\n    lq, gt = dataset[i]","preventionTips":["Regenerate meta_info.txt after any file deletion/move in the dataset","Verify dataset integrity (file count + sizes) once before long training runs","Back up data.mdb/lock.mdb and never build LMDB on a machine that might crash mid-write","Prefer local SSD over flaky NFS mounts for large-scale training"],"tags":["realesrgan","file-io","retry","dataset","corrupted-data"],"backgroundTag":"file-read-io-error","analyzedSha":"a4abfb2979a7bbff3f69f58f58ae324608821e27","analyzedAt":"2026-08-27T03:04:18.378Z","schemaVersion":2},"datasetVersion":"2026-08-27T03:17:27.898Z"}