unslothai/unsloth · error · RuntimeError

A diffusion component is larger than the available GPU memor

Error message

A diffusion component is larger than the available GPU memory, and granular streaming offload could not be enabled: {exc}

What it means

Raised when apply_group_offloading / streaming offload setup throws partway through installing hooks on transformer blocks. This path is only selected after a component measured larger than the safe device budget, meaning model-offload fallback would deterministically OOM — so the failure is fatal and reported verbatim with the underlying exception chained.

Source

Thrown at studio/backend/core/inference/diffusion_memory.py:1229

            if "use_stream" in params:
                kwargs["use_stream"] = use_stream
            if use_stream and "non_blocking" in params:
                kwargs["non_blocking"] = True
            if use_stream and "record_stream" in params:
                kwargs["record_stream"] = False
            if use_stream and "low_cpu_mem_usage" in params:
                kwargs["low_cpu_mem_usage"] = True
            apply_group_offloading(module, **kwargs)
            installed += 1
    except Exception as exc:
        if logger is not None:
            logger.warning(
                "diffusion.memory: granular streaming offload failed after installing hooks "
                "on %d module(s): %s",
                installed,
                exc,
            )
        raise RuntimeError(
            "A diffusion component is larger than the available GPU memory, and granular "
            f"streaming offload could not be enabled: {exc}"
        ) from exc

View on GitHub (pinned to 203007d190)

Solutions

1. Read the chained {exc} — the underlying exception names the actual API/Compat problem
2. Upgrade or pin diffusers/torch to versions where apply_group_offloading is stable
3. Use a more quantized model so the component fits without streaming offload
4. Failing that, free GPU memory or move to a device with more VRAM

try:
    load_diffusion(model, quant=q)
except RuntimeError as e:
    if "granular streaming offload could not be enabled" in str(e):
        # fall back to a more aggressive quantization that fits without streaming
        load_diffusion(model, quant=smaller_quant(q))

When it happens

Trigger: A diffusion component (transformer or text encoder) exceeds the GPU budget, the planner picks granular streaming offload, and the diffusers/torch API call raises — e.g. version mismatch in apply_group_offloading kwargs, CUDA context error mid-hook, or an unexpected module shape.

Common situations: diffusers version drift changing apply_group_offloading's signature; GPU driver/CUDA problems while moving blocks; exotic transformer architectures the offload hooks don't traverse.

Related errors

• apply_chat_template_for_generation: no attempt produced a re
• Local base_repo is not a diffusers pipeline directory (no {i
• Failed to apply LoRA: {exc}
• '{family_name}' needs diffusers ({pipeline_class}), which th
• '{family_name}' needs diffusers ({pipeline_class}), but this

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/d35e5b6fea37a26b. Report an issue: GitHub.