{"record":{"id":"5c1c9006eacf6c06","repo":"unslothai/unsloth","slug":"what-needs-about-required-1024-0f-gb-of-mem","errorCode":null,"errorMessage":"{what} needs about {required / 1024:.0f} GB of memory for its weights, but only about {int(budget) / 1024:.0f} GB is usable on this device ({free_note}). This device has unified memory, so the CPU and GPU share one pool: offloading weights to the CPU frees nothing, and the operating system stops an oversized load outright instead of reporting an out-of-memory error. Use a smaller or more quantized model, free memory by closing other applications, or set {UNIFIED_OVERSIZE_ENV}=1 to attempt the load anyway.","messagePattern":"(.+?) needs about (.+?) GB of memory for its weights, but only about (.+?) GB is usable on this device \\((.+?)\\)\\. This device has unified memory, so the CPU and GPU share one pool: offloading weights to the CPU frees nothing, and the operating system stops an oversized load outright instead of reporting an out-of-memory error\\. Use a smaller or more quantized model, free memory by closing other applications, or set (.+?)=1 to attempt the load anyway\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"studio/backend/core/inference/diffusion_memory.py","lineNumber":513,"sourceCode":"    *,\n    family: Optional[str] = None,\n    logger: Any = None,\n) -> None:\n    \"\"\"Refuse a load whose weights cannot fit unified device memory. No-op on every other\n    placement, so the discrete-VRAM path is untouched.\n\n    Lives outside ``plan_diffusion_memory`` on purpose: the planner is a pure sizing function\n    that both loaders call SPECULATIVELY (the image loader re-plans candidate quantisations, and\n    both re-plan against a settled snapshot), and a planner that raised would turn those probes\n    into load failures instead of letting a smaller candidate win. Call this once, on the plan\n    the loader has committed to, after the previous pipeline has been evicted so the free\n    reading is the memory the load actually gets.\"\"\"\n    message = unified_memory_shortfall_message(plan, family = family)\n    if message is None:\n        return\n    if logger is not None:\n        logger.error(\"diffusion.memory: refusing oversized unified-memory load: %s\", message)\n    raise RuntimeError(message)\n\n\ndef _sum_required(*values: Optional[int]) -> Optional[int]:\n    total = 0\n    for value in values:\n        if value is None:\n            return None\n        total += int(value)\n    return total\n\n\n# ── the planner ───────────────────────────────────────────────────────────────\n\n\ndef plan_diffusion_memory(\n    *,\n    target: Any,\n    device_memory: DeviceMemory,","sourceCodeStart":495,"sourceCodeEnd":531,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/core/inference/diffusion_memory.py#L495-L531","documentation":"A hard refusal raised after the loader commits to a plan: the model's weights need more memory than the device budget on a unified-memory system (e.g. Apple Silicon). Because CPU and GPU share one pool, offload cannot help — the OS would kill the oversized allocation without an OOM report — so the planner refuses the load with an actionable message instead.","triggerScenarios":"Loading a large/under-quantized diffusion model on a Mac or other unified-memory device where required weights MiB exceeds the usable budget. Deliberately raised only after the previous pipeline is evicted, on the committed plan, so the free reading reflects reality.","commonSituations":"Loading an 8B+ transformer or fp16 pipeline on a 16/32 GB Mac; other apps consuming shared memory; user picked no quantization when the model needed it.","solutions":["Pick a smaller or more quantized model/quantization (e.g. GGUF Q4 instead of fp16)","Free memory by closing other applications and retry","Set UNSLOTH_DIFFUSION_ALLOW_OVERSIZED_LOAD=1 to attempt the load anyway (risks an OS-level kill)","If the shortfall is small, verify no stale pipeline is holding memory before retrying"],"exampleFix":"// before\nload_model(\"flux-dev\", quant=None)  # 12 GB weights on 16 GB unified Mac\n// after\nload_model(\"flux-dev\", quant=\"q4_k_m\")  # fits the budget","handlingStrategy":"validation","validationCode":"# Ask the planner speculatively — it returns a shortfall message instead of raising\nmsg = unified_memory_shortfall_message(plan, family=family)\nif msg:\n    # pick a smaller quantization before attempting the load\n    ...","typeGuard":null,"tryCatchPattern":"try:\n    load_diffusion(model, quant=q)\nexcept RuntimeError as e:\n    if \"unified memory\" in str(e):\n        retry_with_smaller_quant()  # or surface to user\n    raise","preventionTips":["On unified-memory devices, plan before loading and choose a quantization that fits","Close memory-heavy apps before big loads","Treat UNSLOTH_DIFFUSION_ALLOW_OVERSIZED_LOAD=1 as a known-risk escape hatch, not a fix"],"tags":["memory","unified-memory","diffusion","planner"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}