huggingface/transformers · error · ValueError

DeepGEMM SF must be 2D or 3D, got {sf.dim()}D

Error message

DeepGEMM SF must be 2D or 3D, got {sf.dim()}D

What it means

Error "DeepGEMM SF must be 2D or 3D, got {sf.dim()}D" thrown in huggingface/transformers.

Source

Thrown at src/transformers/integrations/deepgemm.py:400

    When `expected_mn` is set and the SF's M-dim is smaller (block-quantized
    UE8M0, e.g. DSv4-Flash compressor weights with `(N/128, K/128)` SFs), we
    repeat the SF on the M-axis to per-row before packing — the `(INT, 1, gran_k)`
    DeepGEMM kernel branch is the only UE8M0 path on SM100; for `gran_mn > 1`
    the kernel only handles FP32 SFs and would otherwise reject our INT SF here.
    """
    if sf.dtype == torch.float8_e8m0fnu:
        if expected_mn is not None and sf.size(-2) < expected_mn:
            gran_mn = expected_mn // sf.size(-2)
            sf = sf.repeat_interleave(gran_mn, dim=-2)
        if is_sm100:
            sf = sf.contiguous().view(torch.int32)
        else:
            sf = sf.float()
    elif sf.dtype == torch.float32 and is_sm100:
        sf = _ceil_to_ue8m0(sf)

    if sf.dim() not in (2, 3):
        raise ValueError(f"DeepGEMM SF must be 2D or 3D, got {sf.dim()}D")

    # SM90 dispatch transforms SFA and only checks SFB (`sm90_sfb_check`), which needs
    # an unpadded contiguous layout — DeepGEMM does the MN-major alignment itself.
    if not is_sm100:
        return sf.contiguous()

    mn = sf.size(-2)
    kf = sf.size(-1)
    align_to = 16 // sf.element_size()  # `get_tma_aligned_size`: align(mn, 16 / element_size)
    aligned_mn = -(-mn // align_to) * align_to
    target_strides = (1, aligned_mn) if sf.dim() == 2 else (kf * aligned_mn, 1, aligned_mn)

    if tuple(sf.stride()) == target_strides:
        return sf
    out = torch.empty_strided(sf.shape, target_strides, dtype=sf.dtype, device=sf.device)
    out.copy_(sf)
    return out

View on GitHub (pinned to a597f97485)

Solutions

1. Reshape the DeepGEMM scale-factor tensor to 2D or 3D as expected by the kernel.

When it happens

Trigger: Raised in DeepGEMM scale-factor validation when the SF tensor is neither 2D nor 3D.

Common situations: Malformed or wrongly reshaped FP8 scale tensors in a DeepGEMM-quantized checkpoint.

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/5203588b0eaea842. Report an issue: GitHub.