unslothai/unsloth · error · ImageActivationShortfallError

Generating at {w}x{h}{batch_note} needs about {total / 1024:

Error message

Generating at {w}x{h}{batch_note} needs about {total / 1024:.2f} GB of working memory (including about {max(0, int(base_overhead_mib)) / 1024:.2f} GB of fixed overhead), but only about {int(budget) / 1024:.2f} GB is usable on this device (of the {int(free) / 1024:.2f} GB currently free, after reserving room for fragmentation and other processes). Working memory holds this pass's latents and attention buffers, which cannot be offloaded to the CPU the way weights can, so no memory mode recovers this. {Upload a smaller source image (this workflow takes its output size from the image, not the Resolution setting) | Generate at a smaller resolution}{ or a smaller batch size}, free device memory by closing other applications, or set {OVERSIZED_GENERATE_ENV}=1 to attempt it anyway.

What it means

ImageActivationShortfallError raised when the working memory for one generation pass — latents plus attention buffers plus fixed overhead — exceeds the usable device budget. Unlike weights, working memory cannot be CPU-offloaded, so no memory_mode recovers it. The message adapts: source-driven workflows (img2img-style) are told output size comes from the image, not the Resolution setting.

Source

Thrown at studio/backend/core/inference/diffusion_memory.py:1164

    ``ValueError`` on purpose: ``/images/generate`` maps ValueError to HTTP 400 with the message
    as the reason, so this surfaces as an actionable refusal in the UI. RuntimeError there is
    reserved for the two client-state sentinels (not loaded / cancelled) and otherwise becomes an
    opaque 500."""
    message = image_activation_shortfall_message(
        device_memory = device_memory,
        width = width,
        height = height,
        batch_size = batch_size,
        family = family,
        base_overhead_mib = base_overhead_mib,
        source_driven = source_driven,
    )
    if message is None:
        return
    if logger is not None:
        logger.error("diffusion.memory: refusing oversized generation: %s", message)
    raise ImageActivationShortfallError(message)


def _apply_streaming_offload(pipe: Any, device: str, logger: Any) -> None:
    """Stream transformer blocks and text-encoder leaves without whole-component onloads.

    This is selected only after measuring a component larger than the safe device budget, so a
    model-offload fallback would deterministically OOM. Any setup failure is therefore fatal and
    reports the granular-offload failure directly.
    """
    installed = 0
    try:
        import inspect

        import torch
        from diffusers.hooks import apply_group_offloading

        components = getattr(pipe, "components", {})
        if not isinstance(components, dict):

View on GitHub (pinned to 203007d190)

Solutions

  1. For source-driven workflows, use a smaller source image — output size follows the image, not the Resolution setting
  2. Otherwise generate at a smaller resolution
  3. Reduce batch size
  4. Free device memory by closing other applications / models
  5. As a last resort set UNSLOTH_DIFFUSION_ALLOW_OVERSIZED_GENERATE=1 to attempt it anyway (likely real OOM)

Example fix

// before
img2img(source=Image.open("4k_photo.png"))  # working set exceeds VRAM
// after
img2img(source=Image.open("4k_photo.png").resize((1024, 1024)))
Defensive patterns

Strategy: validation

Validate before calling

# Pre-check working-memory estimate before generating
plan = estimate_generation_memory(w, h, batch, family)
if plan_exceeds_budget(plan, device_memory):
    # downscale source image or reduce batch before calling generate

Try / catch

try:
    generate(...)
except ImageActivationShortfallError as e:
    # message already carries the exact remedy (source image / resolution / batch)
    return JSONResponse(status_code=422, content={"detail": str(e)})

Prevention

When it happens

Trigger: Generating at large width x height or with a large batch on a device whose free memory (minus fragmentation/process reserve) is below the computed working-set estimate; feeding a high-resolution source image to a source-driven workflow.

Common situations: 4K img2img on an 8 GB GPU; batch of 8 at 1024px; another process (or the loaded weights) eating VRAM; user increased Resolution not realizing img2img ignores it.

Related errors

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/b6116e22ca9548f0. Report an issue: GitHub.