unslothai/unsloth · error · ValueError

⚠️ {fail_rate:.0%} of the first {PROBE_SIZE} images failed t

Error message

⚠️ {fail_rate:.0%} of the first {PROBE_SIZE} images failed to download ({probe_fail}/{probe_total}). This dataset has too many broken or unreachable image URLs. Consider using a dataset with embedded images instead.

What it means

Raised during VLM dataset conversion when the initial probe batch (first PROBE_SIZE images) exceeds the allowed failure rate for image downloads. The probe runs before the bulk download to fail fast instead of wasting hours fetching a mostly-broken dataset. The ValueError carries either an LLM-generated friendly warning or the formatted default message and is surfaced via _notify().

Source

Thrown at studio/backend/utils/datasets/format_conversion.py:556

            try:
                from .llm_assist import llm_generate_dataset_warning
                friendly = llm_generate_dataset_warning(
                    issues,
                    dataset_name = dataset_name,
                    modality = "vision",
                    column_names = [image_column, text_column],
                )
            except Exception:
                pass
            msg = friendly or (
                f"⚠️ {fail_rate:.0%} of the first {PROBE_SIZE} images failed to download "
                f"({probe_fail}/{probe_total}). "
                "This dataset has too many broken or unreachable image URLs. "
                "Consider using a dataset with embedded images instead."
            )
            logger.info(msg)
            _notify(msg)
            raise ValueError(msg)

        # Estimate time for remaining samples
        remaining = total - PROBE_SIZE
        estimated_seconds = remaining / throughput if throughput > 0 else 0
        eta_str = _format_eta(estimated_seconds)

        info_msg = (
            f"Downloading {total:,} images ({num_workers} workers, ~{throughput:.1f} img/s). "
            f"Estimated time: ~{eta_str}"
        )
        if probe_fail > 0:
            info_msg += f" | {fail_rate:.0%} broken URLs will be skipped"

        logger.info(
            f"✅ Probe passed: {probe_ok}/{probe_total} ok, {probe_fail} failed ({fail_rate:.0%}), {throughput:.1f} img/s"
        )
        logger.info(f"⏱️ Estimated time for {total:,} samples: ~{eta_str}")
        _notify(info_msg)

View on GitHub (pinned to 203007d190)

Solutions

Use a dataset with embedded images (HF image feature with bytes) instead of URL references, as the message suggests.
Spot-check ~10 URLs from the image column manually (curl) to confirm whether links are dead or blocked by auth/egress.
If URLs need an HF token, ensure HUGGING_FACE_HUB_TOKEN / hf token login is configured before conversion.
If the failure is transient (rate limiting), retry conversion after a delay or with fewer download workers.
Pre-download or rewrite the image column to accessible URLs / local paths, then re-run conversion.

Example fix

# before
ds = load_dataset("some/old-url-dataset")  # image column = dead URLs
converted = convert_image_urls_to_vlm(ds)  # raises ValueError

# after
ds = load_dataset("some/embedded-image-dataset")  # images stored as bytes
converted = convert_image_urls_to_vlm(ds)  # probe passes

Defensive patterns

Strategy: validation

Validate before calling

import urllib.request, random

def probe_image_urls(dataset, image_column, n=10, timeout=10):
    """Return success ratio over a random sample of URLs before conversion."""
    urls = random.sample(list(dataset[image_column]), min(n, len(dataset)))
    ok = 0
    for u in urls:
        if not isinstance(u, str) or not u.startswith(("http://", "https://")):
            continue
        try:
            req = urllib.request.Request(u, method="HEAD")
            if urllib.request.urlopen(req, timeout=timeout).status < 400:
                ok += 1
        except Exception:
            pass
    return ok / max(len(urls), 1)

# before conversion:
# assert probe_image_urls(ds, "image_url") >= 0.8

Try / catch

try:
    converted = convert_image_urls_to_vlm(ds)
except ValueError as e:
    if "failed to download" in str(e):
        # switch to embedded-image dataset or repair URL column
        ds = load_dataset(embedded_variant)
        converted = convert_image_urls_to_vlm(ds)
    else:
        raise

Prevention

Prefer HF datasets with embedded image features (bytes) over URL columns.
Run a 10-URL HEAD probe before launching conversion of large datasets.
Keep HF token configured when datasets link to gated HF-hosted assets.
Pin dataset revisions whose URLs are known-good instead of latest.

When it happens

Trigger: Calling the image-URL-to-VLM conversion path (convert to standard VLM format with an image column of URLs) where >= the threshold fraction of the first PROBE_SIZE URL downloads fail, e.g. dead links, 403s from hotlink protection, DNS failures, or expired signed URLs.

Common situations: Training on scraped image-caption datasets (e.g. old ShareGPT/Laion-style dumps) whose hosted images were deleted; datasets behind authentication (HF gated URLs requiring token); proxies/firewalls blocking outbound requests; rate-limited image CDNs returning 429.

Related errors

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/a045488d77c64545. Report an issue: GitHub.