unslothai/unsloth · error · RuntimeError

{what} needs about {required / 1024:.0f} GB of memory for it

Error message

{what} needs about {required / 1024:.0f} GB of memory for its weights, but only about {int(budget) / 1024:.0f} GB is usable on this device ({free_note}). This device has unified memory, so the CPU and GPU share one pool: offloading weights to the CPU frees nothing, and the operating system stops an oversized load outright instead of reporting an out-of-memory error. Use a smaller or more quantized model, free memory by closing other applications, or set {UNIFIED_OVERSIZE_ENV}=1 to attempt the load anyway.

What it means

A hard refusal raised after the loader commits to a plan: the model's weights need more memory than the device budget on a unified-memory system (e.g. Apple Silicon). Because CPU and GPU share one pool, offload cannot help — the OS would kill the oversized allocation without an OOM report — so the planner refuses the load with an actionable message instead.

Source

Thrown at studio/backend/core/inference/diffusion_memory.py:513

    *,
    family: Optional[str] = None,
    logger: Any = None,
) -> None:
    """Refuse a load whose weights cannot fit unified device memory. No-op on every other
    placement, so the discrete-VRAM path is untouched.

    Lives outside ``plan_diffusion_memory`` on purpose: the planner is a pure sizing function
    that both loaders call SPECULATIVELY (the image loader re-plans candidate quantisations, and
    both re-plan against a settled snapshot), and a planner that raised would turn those probes
    into load failures instead of letting a smaller candidate win. Call this once, on the plan
    the loader has committed to, after the previous pipeline has been evicted so the free
    reading is the memory the load actually gets."""
    message = unified_memory_shortfall_message(plan, family = family)
    if message is None:
        return
    if logger is not None:
        logger.error("diffusion.memory: refusing oversized unified-memory load: %s", message)
    raise RuntimeError(message)


def _sum_required(*values: Optional[int]) -> Optional[int]:
    total = 0
    for value in values:
        if value is None:
            return None
        total += int(value)
    return total


# ── the planner ───────────────────────────────────────────────────────────────


def plan_diffusion_memory(
    *,
    target: Any,
    device_memory: DeviceMemory,

View on GitHub (pinned to 203007d190)

Solutions

Pick a smaller or more quantized model/quantization (e.g. GGUF Q4 instead of fp16)
Free memory by closing other applications and retry
Set UNSLOTH_DIFFUSION_ALLOW_OVERSIZED_LOAD=1 to attempt the load anyway (risks an OS-level kill)
If the shortfall is small, verify no stale pipeline is holding memory before retrying

Example fix

// before
load_model("flux-dev", quant=None)  # 12 GB weights on 16 GB unified Mac
// after
load_model("flux-dev", quant="q4_k_m")  # fits the budget

Defensive patterns

Strategy: validation

Validate before calling

# Ask the planner speculatively — it returns a shortfall message instead of raising
msg = unified_memory_shortfall_message(plan, family=family)
if msg:
    # pick a smaller quantization before attempting the load
    ...

Try / catch

try:
    load_diffusion(model, quant=q)
except RuntimeError as e:
    if "unified memory" in str(e):
        retry_with_smaller_quant()  # or surface to user
    raise

Prevention

On unified-memory devices, plan before loading and choose a quantization that fits
Close memory-heavy apps before big loads
Treat UNSLOTH_DIFFUSION_ALLOW_OVERSIZED_LOAD=1 as a known-risk escape hatch, not a fix

When it happens

Trigger: Loading a large/under-quantized diffusion model on a Mac or other unified-memory device where required weights MiB exceeds the usable budget. Deliberately raised only after the previous pipeline is evicted, on the committed plan, so the free reading reflects reality.

Common situations: Loading an 8B+ transformer or fp16 pipeline on a 16/32 GB Mac; other apps consuming shared memory; user picked no quantization when the model needed it.

Related errors

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/5c1c9006eacf6c06. Report an issue: GitHub.