{"record":{"id":"a16a56a458cf7930","repo":"unslothai/unsloth","slug":"image-cannot-be-decoded-img-name-e-remove","errorCode":null,"errorMessage":"Image cannot be decoded: {img.name} ({e}). Remove or replace the corrupt or zero-byte file before training.","messagePattern":"Image cannot be decoded: (.+?) \\((.+?)\\)\\. Remove or replace the corrupt or zero-byte file before training\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"studio/backend/core/training/diffusion_train_common.py","lineNumber":1444,"sourceCode":"                    caption = \"\"\n                break\n        # 2. metadata row keyed by file name (basename or relative path, as_posix so Windows paths match). A sidecar, even empty, wins.\n        if not sidecar_present:\n            caption = meta_caption.get(img.name) or meta_caption.get(\n                img.relative_to(root).as_posix()\n            )\n        # 3. dreambooth instance prompt for any image still without a caption.\n        if not caption and instance_prompt:\n            caption = instance_prompt\n        if caption:\n            if verify_images:\n                # Reject a corrupt/truncated image now via a cheap PIL header probe: otherwise it passes filename-only discovery, the start route frees the GPU models, and the trainer crashes in Image.open.\n                try:\n                    from PIL import Image\n                    with Image.open(img) as _probe:\n                        _probe.verify()\n                except Exception as e:  # noqa: BLE001 -- corrupt/zero-byte/truncated file\n                    raise ValueError(\n                        f\"Image cannot be decoded: {img.name} ({e}). Remove or replace the \"\n                        f\"corrupt or zero-byte file before training.\"\n                    ) from e\n            pairs.append((str(img), caption))\n\n    if not pairs:\n        raise ValueError(\n            \"No captioned images found. Provide a metadata.jsonl / captions.jsonl, per-image \"\n            \".txt captions, or an instance prompt.\"\n        )\n    return pairs\n\n\n# Families whose trainer has no checkpoint/resume support yet. The shared DiffusionLoraConfig\n# carries save_steps / resume_from_checkpoint for every family, so a loop that implements\n# neither has to say so rather than ignore them.\nCHECKPOINTLESS_FAMILIES: frozenset[str] = frozenset({\"minimax-h3\"})\n","sourceCodeStart":1426,"sourceCodeEnd":1462,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/core/training/diffusion_train_common.py#L1426-L1462","documentation":"With verify_images enabled (the start route enables it), each captioned image is opened with PIL and passed through Image.verify() — a cheap header probe. A corrupt, zero-byte, or truncated file fails the probe and raises ValueError naming the file, so the bad upload is rejected BEFORE the resident GPU models are freed, instead of crashing the spawned trainer mid-run.","triggerScenarios":"A dataset directory containing at least one corrupt image that also has a caption (from metadata.jsonl, a sidecar .txt, or an instance_prompt): truncated uploads, zero-byte files from aborted transfers, or files with a mismatched extension.","commonSituations":"Interrupted uploads, files renamed from .png to .jpg without conversion, cloud sync placeholders, or images that preview in some viewers but fail strict decoding.","solutions":["Remove or replace the named file and restart training.","Pre-scan the dataset locally: open every image with PIL and run verify() before uploading.","Re-upload the dataset if the corruption came from an interrupted transfer."],"exampleFix":"# before: dataset contains a zero-byte img_0042.jpg\n# after: pre-scan and drop corrupt files\nfrom pathlib import Path\nfrom PIL import Image\nfor p in Path(data_dir).iterdir():\n    if p.suffix.lower() in {'.png', '.jpg', '.jpeg', '.webp'}:\n        try:\n            with Image.open(p) as im:\n                im.verify()\n        except Exception:\n            p.unlink()  # or move aside and fix","handlingStrategy":"validation","validationCode":"from pathlib import Path\nfrom PIL import Image\nBAD = []\nfor p in Path(data_dir).iterdir():\n    if p.suffix.lower() in {'.png', '.jpg', '.jpeg', '.webp', '.bmp'}:\n        try:\n            with Image.open(p) as im:\n                im.verify()\n        except Exception:\n            BAD.append(p)\nif BAD:\n    raise ValueError(f'corrupt images: {[b.name for b in BAD]}')","typeGuard":null,"tryCatchPattern":"try:\n    pairs = build_caption_pairs(data_dir, verify_images=True, ...)\nexcept ValueError as e:\n    if 'cannot be decoded' in str(e):\n        # name the file to the user for removal/re-upload\n        report_bad_upload(str(e))","preventionTips":["Run a PIL verify() pass on datasets at upload time, not at training time.","Keep verify_images=True on the start route — it protects against crashing after GPU eviction."],"tags":["training","dataset","corrupt-file","pil","preflight"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}