unslothai/unsloth · error · RuntimeError

transformer_quant='{requested}' could not be used: {reason}.

Error message

transformer_quant='{requested}' could not be used: {reason}. Choose Auto to let the backend pick the fastest precision this host can run, or Off to run the DiT at bf16.

What it means

The transformer_quant gate refuses an explicitly pinned quantization scheme when the selector returns None for it after a smoke probe: the family's measured deny list blocks it, torchao cannot run here, or the GPU lacks the needed kernels. The message deliberately offers Auto (let the backend pick) or Off (bf16 DiT). It fires before the heavy load so users do not discover the refusal after a teardown.

Source

Thrown at studio/backend/core/inference/video.py:296

                f"'{normalize_memory_mode(memory_mode)}' memory places the DiT under CPU "
                "offload, and torchao quantised tensors cannot be moved by the offload hooks"
            )
        elif (
            select_transformer_quant_scheme(
                target,
                pinned,
                family = getattr(fam, "name", None),
                # Pre-eviction: an OOM in the smoke probe is the resident model, not the scheme.
                unproven_ok = True,
            )
            is None
        ):
            # An explicit scheme is never swapped for another, so a None means the family's
            # measured deny list, a torchao that cannot run here, or a GPU without the kernels.
            # They read the same to the selector and need different fixes from the user.
            reason = explain_unusable_scheme(getattr(fam, "name", None), pinned)
        if reason is not None:
            raise RuntimeError(
                precision_refusal_message(
                    "transformer_quant", pinned, reason, off_label = "Off to run the DiT at bf16"
                )
            )
    # The mode the loader will ACTUALLY attempt, not the raw request: an explicit int8
    # on a family with no keep-bf16 schedule is rewritten to layerwise fp8 before
    # support is consulted, and that path needs no torchao. Refusing on the raw int8
    # rejected loads the runtime would run and report as fell_back -- Windows ROCm,
    # where the torchao stub kills int8 while fp8 still works.
    # A family with a HOSTED quantized conditioner never touches the generic path this gate
    # reasons about. Studio loads that artifact itself: INT8 storage, a Hadamard rotation and an
    # ordinary F.linear, no torchao and no fp8 tensor cores. Left to the code below, the request is
    # first rewritten int8 -> fp8 by effective_te_quant (H3 has no keep-bf16 schedule) and then
    # refused for want of hardware neither the rewrite nor the real loader needs, so a supported
    # CPU load comes back as a 409. Whether the artifact suits THIS base is decided at the seed,
    # which is not a host-level impossibility and so is not this gate's question.
    # PIPELINE loads only. The hosted conditioner is a Diffusers-path artifact; the native GGUF
    # path picks its Q2/Q4 encoder off the denoiser filename and discards text_encoder_quant

View on GitHub (pinned to 203007d190)

Solutions

Set transformer_quant='auto' (or the UI's Auto) to let the backend pick the fastest precision the host can actually run.
Set it to Off to run the DiT at bf16.
Install/upgrade the torchao build matching your torch and GPU, then retry the pinned scheme.
Check the family's measured deny list — a previous OOM during the smoke probe can deny a scheme that hardware alone would support.

Example fix

# before
load_video_model(repo_id=..., transformer_quant='fp8')  # refused on this host

# after
load_video_model(repo_id=..., transformer_quant='auto')
# or explicit bf16:
load_video_model(repo_id=..., transformer_quant='off')

Defensive patterns

Strategy: fallback

Validate before calling

reason = explain_unusable_scheme(family_name, pinned_scheme) if pinned_scheme not in (None, 'auto') else None
if reason is not None:
    pinned_scheme = 'auto'  # or 'off' for bf16

Try / catch

try:
    load_video_model(repo_id=r, transformer_quant='int8')
except RuntimeError as e:
    if 'transformer_quant' in str(e):
        load_video_model(repo_id=r, transformer_quant='auto')  # backend picks best
    else:
        raise

Prevention

Default to Auto in configs shared across heterogeneous hosts.
Keep torchao versions pinned to builds matching your torch/CUDA/ROCm stack.
Remember a prior OOM smoke probe can deny-list a scheme on this family; clearing measured state may restore it.

When it happens

Trigger: Passing transformer_quant='int8'/'fp8'/etc. explicitly on a host where explain_unusable_scheme reports it unusable — e.g. Windows ROCm where the torchao stub kills int8, a family with the scheme on its deny list after OOM probes, or a GPU without fp8/NVFP4 tensor cores.

Common situations: Copying a config that worked on an NVIDIA datacenter GPU to a consumer or AMD card; pinning int8 on a family whose measured deny list excludes it; torchao not installed or an incompatible version for the pinned mode.

Related errors

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/7536306896de5f26. Report an issue: GitHub.