huggingface/transformers · error · RuntimeError

scale_fmt='ue8m0' requires torch.float8_e8m0fnu, which is on

Error message

scale_fmt='ue8m0' requires torch.float8_e8m0fnu, which is only available in PyTorch >= 2.7 (found {torch.__version__}). Upgrade torch to use UE8M0 FP8 checkpoints.

What it means

Error "scale_fmt='ue8m0' requires torch.float8_e8m0fnu, which is only available in PyTorch >= 2.7 (found {torch.__version__}). Upgrade torch to use UE8M0 FP8 checkpoints." thrown in huggingface/transformers.

Source

Thrown at src/transformers/integrations/finegrained_fp8.py:59

logger = logging.get_logger(__name__)


_FP8_DTYPE = torch.float8_e4m3fn
_FP8_MIN = torch.finfo(_FP8_DTYPE).min
_FP8_MAX = torch.finfo(_FP8_DTYPE).max


@functools.cache
def _get_ue8m0_dtype() -> torch.dtype:
    """Return ``torch.float8_e8m0fnu`` or raise a clear error on torch without FP8 support.

    UE8M0 scales are always stored/consumed as this single dtype — the kernels (Triton
    finegrained + DeepGEMM) read it natively, and supporting the same scales in mixed
    container dtypes would be a mess — so fail loudly rather than fall back."""
    if not hasattr(torch, "float8_e8m0fnu"):
        raise RuntimeError(
            "scale_fmt='ue8m0' requires torch.float8_e8m0fnu, which is only available in "
            f"PyTorch >= 2.7 (found {torch.__version__}). Upgrade torch to use UE8M0 FP8 checkpoints."
        )
    return torch.float8_e8m0fnu


def _first_attr(obj, *names):
    for name in names:
        if hasattr(obj, name):
            return getattr(obj, name)
    raise AttributeError(f"{type(obj).__name__} has none of: {names}")


@dataclass(frozen=True)
class FineGrainedFP8:
    """Entry points exposed by the `kernels-community/finegrained-fp8` Triton kernel."""

    matmul: Callable

View on GitHub (pinned to a597f97485)

Solutions

1. Upgrade PyTorch to >= 2.7 to get torch.float8_e8m0fnu.
2. Use a checkpoint with scale_fmt='float' instead of 'ue8m0'.

When it happens

Trigger: Raised in finegrained FP8 integration when scale_fmt='ue8m0' is used with PyTorch < 2.7 lacking float8_e8m0fnu.

Common situations: Loading a UE8M0 FP8 checkpoint on an older torch version without the e8m0 dtype.

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/cf49d7e0cd32f15c. Report an issue: GitHub.