{"record":{"id":"2d6354c47940690f","repo":"huggingface/transformers","slug":"deepgemm-has-no-float32-scale-factor-path-on-black","errorCode":null,"errorMessage":"DeepGEMM has no float32 scale-factor path on Blackwell (SM100): these scales are plain float32 (quantization_config.scale_fmt='float'), and rounding them to UE8M0 would silently corrupt the output. Use a checkpoint quantized with scale_fmt='ue8m0', or a path that consumes float32 block scales directly — the FP8 linear falls back to Triton automatically; for experts use `model.set_experts_implementation('grouped_mm')`.","messagePattern":"DeepGEMM has no float32 scale-factor path on Blackwell \\(SM100\\): these scales are plain float32 \\(quantization_config\\.scale_fmt='float'\\), and rounding them to UE8M0 would silently corrupt the output\\. Use a checkpoint quantized with scale_fmt='ue8m0', or a path that consumes float32 block scales directly — the FP8 linear falls back to Triton automatically; for experts use `model\\.set_experts_implementation\\('grouped_mm'\\)`\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"src/transformers/integrations/deepgemm.py","lineNumber":338,"sourceCode":"    Uses `is_sm100()` (compile-safe via `assume_constant_result`), so the whole guard folds to a constant under\n    ``torch.compile``: the valid case compiles away to nothing, while an unsupported combo fails loud\n    rather than letting the hot path silently corrupt (unlike an ``is_compiling`` skip, which would miss a\n    model compiled from cold with no eager warmup). Both raise ``NotImplementedError``, which\n    ``fp8_linear`` treats as \"DeepGEMM declined\" and falls back to Triton (SM90 consuming float32 SFs\n    directly is fine, so those cases are no-ops).\n    \"\"\"\n    if not is_sm100():\n        # SM90: DeepGEMM has no FP4 (int8-packed) kernel, but consumes float32 SFs directly.\n        if weight.dtype == torch.int8:\n            raise NotImplementedError(\n                \"DeepGEMM's FP4 (int8-packed) path requires a Blackwell (SM100+) GPU; FP4 weights have no \"\n                \"Hopper (SM90) kernel. Use an FP8 checkpoint, or run on a Blackwell GPU.\"\n            )\n        return\n\n    # SM100: DeepGEMM has no float32 scale-factor path.\n    if scale.dtype == torch.float32:\n        raise NotImplementedError(\n            \"DeepGEMM has no float32 scale-factor path on Blackwell (SM100): these scales are plain float32 \"\n            \"(quantization_config.scale_fmt='float'), and rounding them to UE8M0 would silently corrupt the \"\n            \"output. Use a checkpoint quantized with scale_fmt='ue8m0', or a path that consumes float32 block \"\n            \"scales directly — the FP8 linear falls back to Triton automatically; for experts use \"\n            \"`model.set_experts_implementation('grouped_mm')`.\"\n        )\n\n\ndef _ceil_to_ue8m0(sf: torch.Tensor) -> torch.Tensor:\n    \"\"\"Round each fp32 SF up to the nearest power of 2 (zero mantissa).\n\n    Mirrors `deep_gemm.utils.math.ceil_to_ue8m0`. On SM100 the kernel's\n    `pack_fp32_into_ue8m0` cleanly extracts the biased exponent only when the\n    mantissa is already zero — its inner shifts (`>> 15`, `>> 7`, `<< 1`)\n    otherwise leak mantissa bits into adjacent UE8M0 byte slots and silently\n    corrupt the SF. SM90 consumes raw fp32 SFs without going through this path.\n    \"\"\"\n    int_view = sf.view(torch.int32)","sourceCodeStart":320,"sourceCodeEnd":356,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/integrations/deepgemm.py#L320-L356","documentation":"Error \"DeepGEMM has no float32 scale-factor path on Blackwell (SM100): these scales are plain float32 (quantization_config.scale_fmt='float'), and rounding them to UE8M0 would silently corrupt the output. Use a checkpoint quantized with scale_fmt='ue8m0', or a path that consumes float32 block scales directly — the FP8 linear falls back to Triton automatically; for experts use `model.set_experts_implementation('grouped_mm')`.\" thrown in huggingface/transformers.","triggerScenarios":"Raised in DeepGEMM FP8 path on Blackwell when scale factors are plain float32 instead of UE8M0.","commonSituations":"Running a scale_fmt='float' FP8 checkpoint on SM100 hardware where only UE8M0 scales are supported.","solutions":["Use a checkpoint quantized with scale_fmt='ue8m0'.","Call `model.set_experts_implementation('grouped_mm')` for experts so float32 scales are consumed directly."],"exampleFix":null,"handlingStrategy":null,"validationCode":null,"typeGuard":null,"tryCatchPattern":null,"preventionTips":[],"tags":[],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}