unslothai/unsloth · error · ValueError

Unsupported transformer_quant '{value}'. Use one of: {', '.j

Error message

Unsupported transformer_quant '{value}'. Use one of: {', '.join(TQ_MODES)}.

What it means

normalize_transformer_quant validates the transformer_quant parameter: None/''/'none'/'off' collapse to None, otherwise the value must be in TQ_MODES ('auto' plus the supported transformer quant schemes). Unknown values raise ValueError for cheap 4xx rejection before any model loading.

Source

Thrown at studio/backend/core/inference/diffusion_transformer_quant.py:437

    if "GEFORCE" in name or "TITAN" in name:
        return True
    if any(marker in name for marker in _PROFESSIONAL_GPU_MARKERS):
        return False
    tokens = set(re.split(r"[^A-Z0-9]+", name))
    return not (tokens & _DATACENTER_GPU_TOKENS)


def normalize_transformer_quant(value: Optional[str]) -> Optional[str]:
    """Lower/strip a requested transformer quant; None / "" / "none" / "off" -> None.

    Raises ValueError for an unsupported value so a bad request is rejected cheaply."""
    if value is None:
        return None
    normalized = str(value).strip().lower().replace("-", "_")
    if not normalized or normalized in ("none", "off"):
        return None
    if normalized not in TQ_MODES:
        raise ValueError(
            f"Unsupported transformer_quant '{value}'. Use one of: {', '.join(TQ_MODES)}."
        )
    return normalized


def dense_transformer_supported(target: Any) -> bool:
    """Whether the dense-source quant path is usable for ``target``: a CUDA device with bf16
    dtype (the only config any torchao dynamic scheme accelerates). Cheap loader pre-check."""
    if getattr(target, "device", None) != "cuda":
        return False
    # The Windows-ROCm torchao stub's quantize_ is a no-op, so the smoke probe passes on a
    # still-dense Linear and the transformer gets MARKED quantised without being quantised,
    # giving the wrong VRAM budget and compile policy.
    if is_stubbed("torchao"):
        return False
    try:
        import torch
        return getattr(target, "dtype", None) is torch.bfloat16

View on GitHub (pinned to 203007d190)

Solutions

Send 'auto' to let the loader pick, or a scheme listed in the error message
Send 'none'/'off' or omit the field to keep the transformer dense
If you need a specific scheme (e.g. fp8), confirm the backend build exports it in TQ_MODES

Example fix

// before
{"transformer_quant": "int8"}
// after
{"transformer_quant": "auto"}

Defensive patterns

Strategy: validation

Validate before calling

# Reflect the build's accepted modes rather than hardcoding
from core.inference.diffusion_transformer_quant import TQ_MODES

def valid_tq(v) -> bool:
    if v is None:
        return True
    n = str(v).strip().lower().replace("-", "_")
    return n in ("", "none", "off") or n in TQ_MODES

Type guard

def is_transformer_quant(v, allowed: set[str]) -> bool:
    return v is None or (isinstance(v, str) and str(v).strip().lower().replace("-", "_") in allowed | {"", "none", "off"})

Try / catch

try:
    normalize_transformer_quant(req.transformer_quant)
except ValueError as e:
    return JSONResponse(status_code=400, content={"detail": str(e)})

Prevention

Use 'auto' unless a specific scheme is required
Derive the allowed set from the running backend, not from docs of another version
Don't mirror text_encoder_quant values into transformer_quant

When it happens

Trigger: Request with transformer_quant set to a scheme this build doesn't support — e.g. 'int8' when TQ_MODES only covers gguf-style/auto schemes, or a typo like 'fp8_dyn'.

Common situations: Reusing text_encoder_quant vocabulary for transformer_quant; client/server version mismatch; schemes gated behind optional torchao not listed in this build's TQ_MODES.

Related errors

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/d0f696cf9c01c19b. Report an issue: GitHub.