unslothai/unsloth · error · ValueError

⚠️ {fail_rate:.0%} of images failed to download ({failed_cou

Error message

⚠️ {fail_rate:.0%} of images failed to download ({failed_count}/{total}). This dataset has too many broken or unreachable image URLs. Consider using a dataset with embedded images instead.

What it means

Raised after the bulk image download phase of VLM dataset conversion when the overall failure rate across all samples exceeds the allowed threshold. Unlike the probe-stage error, this fires once the full (or remainder) download has run and too many URLs failed. It aborts the conversion with a ValueError containing either an LLM-friendly warning or the default formatted message via _notify().

Source

Thrown at studio/backend/utils/datasets/format_conversion.py:673

            ]
            friendly = None
            try:
                from .llm_assist import llm_generate_dataset_warning
                friendly = llm_generate_dataset_warning(
                    issues,
                    dataset_name = dataset_name,
                    modality = "vision",
                    column_names = [image_column, text_column],
                )
            except Exception:
                pass
            msg = friendly or (
                f"⚠️ {fail_rate:.0%} of images failed to download ({failed_count}/{total}). "
                "This dataset has too many broken or unreachable image URLs. "
                "Consider using a dataset with embedded images instead."
            )
            _notify(msg)
            raise ValueError(msg)

    if len(converted_list) == 0:
        issues = [
            f"All {total} samples failed during VLM conversion — no usable images found",
            f"Image column '{image_column}' may contain URLs that are no longer accessible, "
            "or local file paths that don't exist",
        ]
        friendly = None
        try:
            from .llm_assist import llm_generate_dataset_warning
            friendly = llm_generate_dataset_warning(
                issues,
                dataset_name = dataset_name,
                modality = "vision",
                column_names = [image_column, text_column],
            )
        except Exception:
            pass

View on GitHub (pinned to 203007d190)

Solutions

Re-run conversion with fewer download workers to avoid CDN rate limiting (worker count is shown in the info message before download).
Check network egress / proxy settings; a proxy that drops sustained parallel connections produces exactly this pattern.
Filter the dataset first: probe URLs with a small HEAD-request script, keep only reachable rows, then convert.
Switch to a dataset with embedded image bytes (HF datasets with an image feature) to avoid network dependency entirely.
If links are signed/expired, re-generate the URL column from the source dataset revision.

Example fix

# before
result = convert_dataset(images_url_ds, num_workers=32)  # mass 429s -> ValueError

# after
result = convert_dataset(images_url_ds, num_workers=4)  # stays under rate limit

Defensive patterns

Strategy: retry

Validate before calling

def safe_convert(ds, image_column, worker_steps=(16, 8, 2)):
    last = None
    for workers in worker_steps:
        try:
            return convert_dataset(ds, image_column=image_column, num_workers=workers)
        except ValueError as e:
            if "failed to download" not in str(e):
                raise
            last = e
    raise last

Try / catch

try:
    result = convert_dataset(ds, image_column="image_url")
except ValueError as e:
    if "too many broken" in str(e):
        log.warning("Image host throttling suspected; retrying with 2 workers")
        result = convert_dataset(ds, image_column="image_url", num_workers=2)
    else:
        raise

Prevention

Start bulk image downloads with conservative worker counts to avoid CDN 429s.
Monitor the 'X% broken URLs will be skipped' notice mid-run and abort early if it climbs.
Cache downloaded images locally keyed by URL hash so reruns skip network.
Keep converted snapshots of URL datasets so a one-time success is reusable.

When it happens

Trigger: Running VLM conversion over a dataset whose image column contains URLs, where the probe batch passed but the aggregate failed_count/total ratio crosses the rejection threshold (e.g. links that die mid-download, CDN throttling after N requests, mixed good/dead URL batches).

Common situations: Large datasets scraped long ago with progressive link rot; image hosts that start returning 403/429 after bulk fetching; flaky networks where sustained throughput drops cause mass timeouts; datasets mixing accessible and expired signed URLs.

Related errors

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/6c7a037a170448dd. Report an issue: GitHub.