{"record":{"id":"3832060b4158c5d4","repo":"huggingface/pytorch-image-models","slug":"invalid-or-corrupt-tar-info-cache-file-cache-path","errorCode":null,"errorMessage":"Invalid or corrupt tar info cache file {cache_path}.","messagePattern":"Invalid or corrupt tar info cache file (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"timm/data/readers/reader_image_in_tar.py","lineNumber":115,"sourceCode":"        tar_filenames = glob(os.path.join(root, '*.tar'), recursive=True)\n    num_tars = len(tar_filenames)\n    tar_bytes = sum([os.path.getsize(f) for f in tar_filenames])\n    assert num_tars, f'No .tar files found at specified path ({root}).'\n\n    _logger.info(f'Scanning {tar_bytes/1024**2:.2f}MB of tar files...')\n    info = dict(tartrees=[])\n    cache_path = ''\n    if cache_tarinfo is None:\n        cache_tarinfo = True if tar_bytes > 10*1024**3 else False  # FIXME magic number, 10GB\n    if cache_tarinfo:\n        cache_filename = '_' + root_name + CACHE_FILENAME_SUFFIX\n        cache_path = os.path.join(root, cache_filename)\n    if os.path.exists(cache_path):\n        _logger.info(f'Reading tar info from cache file {cache_path}.')\n        with open(cache_path, 'rb') as pf:\n            info = _TarInfoUnpickler(pf).load()\n        if not isinstance(info, dict) or not isinstance(info.get('tartrees'), list):\n            raise ValueError(f'Invalid or corrupt tar info cache file {cache_path}.')\n        assert len(info['tartrees']) == num_tars, \"Cached tartree len doesn't match number of tarfiles\"\n    else:\n        for i, fn in enumerate(tar_filenames):\n            path = '' if root_is_tar else os.path.splitext(os.path.basename(fn))[0]\n            with tarfile.open(fn, mode='r|') as tf:  # tarinfo scans done in streaming mode\n                parent_info = dict(name=os.path.relpath(fn, root), path=path, ti=None, children=[], samples=[])\n                num_samples = _extract_tarinfo(tf, parent_info, extensions=extensions)\n                num_children = len(parent_info[\"children\"])\n                _logger.debug(\n                    f'{i}/{num_tars}. Extracted tarinfos from {fn}. {num_children} children, {num_samples} samples.')\n            info['tartrees'].append(parent_info)\n        if cache_path:\n            _logger.info(f'Writing tar info to cache file {cache_path}.')\n            with open(cache_path, 'wb') as pf:\n                pickle.dump(info, pf)\n\n    samples = []\n    labels = []","sourceCodeStart":97,"sourceCodeEnd":133,"githubUrl":"https://github.com/huggingface/pytorch-image-models/blob/9a5261e31b3b5128526eb2658333b4c0a54464ae/timm/data/readers/reader_image_in_tar.py#L97-L133","documentation":"When a timm tar dataset directory has a cached tar-info pickle (e.g. tartrees.pkl), it is loaded to avoid rescanning the tars; this error means the unpickled object is not the expected {'tartrees': [...]} dict structure, i.e. the cache is corrupt or from an incompatible version.","triggerScenarios":"Constructing ImageDataset/ReaderInTar on a tar-root directory where the cache file exists but was truncated, written by another tool, or is a stale format.","commonSituations":"A cache write was interrupted (disk full, killed process); the directory previously held different tar contents and the cache is stale; format changes between timm versions.","solutions":["Delete the cache file (default name like _tartrees.pkl / cache_filename in the tar root) so it is rebuilt by rescanning the tars.","If it recurs, check tar file integrity (tar -tf each shard) and disk space.","Re-run dataset creation to regenerate a fresh cache."],"exampleFix":"rm /data/train_imagenet/tartrees.pkl\n# then re-run dataset creation; tars are rescanned and cache rebuilt","handlingStrategy":"fallback","validationCode":"import os\nfrom timm.data.readers.reader_image_in_tar import CACHE_FILE_NAME as _  # module-specific name may vary\ncache = os.path.join(root, cache_filename)\ntry:\n    ds = ImageDataset(root)  # in_tar inferred from tar presence\nexcept ValueError as e:\n    if 'cache file' in str(e) and os.path.exists(cache):\n        os.remove(cache)\n        ds = ImageDataset(root)","typeGuard":null,"tryCatchPattern":"try:\n    reader = ReaderInTar(root)\nexcept ValueError as e:\n    if 'corrupt tar info cache' in str(e):\n        os.remove(cache_path); reader = ReaderInTar(root)\n    else:\n        raise","preventionTips":["Treat tar-info caches as disposable build artifacts.","Regenerate caches after changing tar contents.","Don't share cache files across timm versions."],"tags":["timm","tar-dataset","cache-corruption"],"backgroundTag":"corrupt-cache-file","analyzedSha":"9a5261e31b3b5128526eb2658333b4c0a54464ae","analyzedAt":"2026-08-27T02:34:25.417Z","schemaVersion":2},"datasetVersion":"2026-08-27T03:17:27.898Z"}