unslothai/unsloth · error · ValueError

Image cannot be decoded: {img.name} ({e}). Remove or replace

Error message

Image cannot be decoded: {img.name} ({e}). Remove or replace the corrupt or zero-byte file before training.

What it means

With verify_images enabled (the start route enables it), each captioned image is opened with PIL and passed through Image.verify() — a cheap header probe. A corrupt, zero-byte, or truncated file fails the probe and raises ValueError naming the file, so the bad upload is rejected BEFORE the resident GPU models are freed, instead of crashing the spawned trainer mid-run.

Source

Thrown at studio/backend/core/training/diffusion_train_common.py:1444

                    caption = ""
                break
        # 2. metadata row keyed by file name (basename or relative path, as_posix so Windows paths match). A sidecar, even empty, wins.
        if not sidecar_present:
            caption = meta_caption.get(img.name) or meta_caption.get(
                img.relative_to(root).as_posix()
            )
        # 3. dreambooth instance prompt for any image still without a caption.
        if not caption and instance_prompt:
            caption = instance_prompt
        if caption:
            if verify_images:
                # Reject a corrupt/truncated image now via a cheap PIL header probe: otherwise it passes filename-only discovery, the start route frees the GPU models, and the trainer crashes in Image.open.
                try:
                    from PIL import Image
                    with Image.open(img) as _probe:
                        _probe.verify()
                except Exception as e:  # noqa: BLE001 -- corrupt/zero-byte/truncated file
                    raise ValueError(
                        f"Image cannot be decoded: {img.name} ({e}). Remove or replace the "
                        f"corrupt or zero-byte file before training."
                    ) from e
            pairs.append((str(img), caption))

    if not pairs:
        raise ValueError(
            "No captioned images found. Provide a metadata.jsonl / captions.jsonl, per-image "
            ".txt captions, or an instance prompt."
        )
    return pairs


# Families whose trainer has no checkpoint/resume support yet. The shared DiffusionLoraConfig
# carries save_steps / resume_from_checkpoint for every family, so a loop that implements
# neither has to say so rather than ignore them.
CHECKPOINTLESS_FAMILIES: frozenset[str] = frozenset({"minimax-h3"})

View on GitHub (pinned to 203007d190)

Solutions

Remove or replace the named file and restart training.
Pre-scan the dataset locally: open every image with PIL and run verify() before uploading.
Re-upload the dataset if the corruption came from an interrupted transfer.

Example fix

# before: dataset contains a zero-byte img_0042.jpg
# after: pre-scan and drop corrupt files
from pathlib import Path
from PIL import Image
for p in Path(data_dir).iterdir():
    if p.suffix.lower() in {'.png', '.jpg', '.jpeg', '.webp'}:
        try:
            with Image.open(p) as im:
                im.verify()
        except Exception:
            p.unlink()  # or move aside and fix

Defensive patterns

Strategy: validation

Validate before calling

from pathlib import Path
from PIL import Image
BAD = []
for p in Path(data_dir).iterdir():
    if p.suffix.lower() in {'.png', '.jpg', '.jpeg', '.webp', '.bmp'}:
        try:
            with Image.open(p) as im:
                im.verify()
        except Exception:
            BAD.append(p)
if BAD:
    raise ValueError(f'corrupt images: {[b.name for b in BAD]}')

Try / catch

try:
    pairs = build_caption_pairs(data_dir, verify_images=True, ...)
except ValueError as e:
    if 'cannot be decoded' in str(e):
        # name the file to the user for removal/re-upload
        report_bad_upload(str(e))

Prevention

Run a PIL verify() pass on datasets at upload time, not at training time.
Keep verify_images=True on the start route — it protects against crashing after GPU eviction.

When it happens

Trigger: A dataset directory containing at least one corrupt image that also has a caption (from metadata.jsonl, a sidecar .txt, or an instance_prompt): truncated uploads, zero-byte files from aborted transfers, or files with a mismatched extension.

Common situations: Interrupted uploads, files renamed from .png to .jpg without conversion, cloud sync placeholders, or images that preview in some viewers but fail strict decoding.

Related errors

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/a16a56a458cf7930. Report an issue: GitHub.