{"record":{"id":"d35e5b6fea37a26b","repo":"unslothai/unsloth","slug":"a-diffusion-component-is-larger-than-the-available","errorCode":null,"errorMessage":"A diffusion component is larger than the available GPU memory, and granular streaming offload could not be enabled: {exc}","messagePattern":"A diffusion component is larger than the available GPU memory, and granular streaming offload could not be enabled: (.+?)","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"studio/backend/core/inference/diffusion_memory.py","lineNumber":1229,"sourceCode":"            if \"use_stream\" in params:\n                kwargs[\"use_stream\"] = use_stream\n            if use_stream and \"non_blocking\" in params:\n                kwargs[\"non_blocking\"] = True\n            if use_stream and \"record_stream\" in params:\n                kwargs[\"record_stream\"] = False\n            if use_stream and \"low_cpu_mem_usage\" in params:\n                kwargs[\"low_cpu_mem_usage\"] = True\n            apply_group_offloading(module, **kwargs)\n            installed += 1\n    except Exception as exc:\n        if logger is not None:\n            logger.warning(\n                \"diffusion.memory: granular streaming offload failed after installing hooks \"\n                \"on %d module(s): %s\",\n                installed,\n                exc,\n            )\n        raise RuntimeError(\n            \"A diffusion component is larger than the available GPU memory, and granular \"\n            f\"streaming offload could not be enabled: {exc}\"\n        ) from exc\n","sourceCodeStart":1211,"sourceCodeEnd":1233,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/core/inference/diffusion_memory.py#L1211-L1233","documentation":"Raised when apply_group_offloading / streaming offload setup throws partway through installing hooks on transformer blocks. This path is only selected after a component measured larger than the safe device budget, meaning model-offload fallback would deterministically OOM — so the failure is fatal and reported verbatim with the underlying exception chained.","triggerScenarios":"A diffusion component (transformer or text encoder) exceeds the GPU budget, the planner picks granular streaming offload, and the diffusers/torch API call raises — e.g. version mismatch in apply_group_offloading kwargs, CUDA context error mid-hook, or an unexpected module shape.","commonSituations":"diffusers version drift changing apply_group_offloading's signature; GPU driver/CUDA problems while moving blocks; exotic transformer architectures the offload hooks don't traverse.","solutions":["Read the chained {exc} — the underlying exception names the actual API/Compat problem","Upgrade or pin diffusers/torch to versions where apply_group_offloading is stable","Use a more quantized model so the component fits without streaming offload","Failing that, free GPU memory or move to a device with more VRAM"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try:\n    load_diffusion(model, quant=q)\nexcept RuntimeError as e:\n    if \"granular streaming offload could not be enabled\" in str(e):\n        # fall back to a more aggressive quantization that fits without streaming\n        load_diffusion(model, quant=smaller_quant(q))","preventionTips":["Pin diffusers/torch versions known-good for apply_group_offloading","Prefer quantization levels where components fit without streaming offload","Log the chained __cause__ — it names the actual API failure"],"tags":["memory","offload","diffusers","compatibility"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}