{"record":{"id":"bf66defacebf2d48","repo":"huggingface/transformers","slug":"deepgemm-s-fp4-int8-packed-path-requires-a-black","errorCode":null,"errorMessage":"DeepGEMM's FP4 (int8-packed) path requires a Blackwell (SM100+) GPU; FP4 weights have no Hopper (SM90) kernel. Use an FP8 checkpoint, or run on a Blackwell GPU.","messagePattern":"DeepGEMM's FP4 \\(int8-packed\\) path requires a Blackwell \\(SM100\\+\\) GPU; FP4 weights have no Hopper \\(SM90\\) kernel\\. Use an FP8 checkpoint, or run on a Blackwell GPU\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"src/transformers/integrations/deepgemm.py","lineNumber":330,"sourceCode":"    \"\"\"Before-load guard for DeepGEMM's FP8/FP4 arch constraints on the given weight/scale dtypes:\n\n      - FP4 (``int8``-packed) weights have no Hopper (SM90) kernel — they need Blackwell (SM100+).\n      - Blackwell has no float32 scale-factor path: ``_coerce_sf_for_kernel`` would silently round a\n        ``float32`` scale to UE8M0 and corrupt the output. UE8M0 scales load as ``float8_e8m0fnu`` (the\n        loader normalizes even float32-container checkpoints like dsv4-flash-base), so a ``float32`` scale\n        on SM100 means a genuine non-UE8M0 checkpoint.\n\n    Uses `is_sm100()` (compile-safe via `assume_constant_result`), so the whole guard folds to a constant under\n    ``torch.compile``: the valid case compiles away to nothing, while an unsupported combo fails loud\n    rather than letting the hot path silently corrupt (unlike an ``is_compiling`` skip, which would miss a\n    model compiled from cold with no eager warmup). Both raise ``NotImplementedError``, which\n    ``fp8_linear`` treats as \"DeepGEMM declined\" and falls back to Triton (SM90 consuming float32 SFs\n    directly is fine, so those cases are no-ops).\n    \"\"\"\n    if not is_sm100():\n        # SM90: DeepGEMM has no FP4 (int8-packed) kernel, but consumes float32 SFs directly.\n        if weight.dtype == torch.int8:\n            raise NotImplementedError(\n                \"DeepGEMM's FP4 (int8-packed) path requires a Blackwell (SM100+) GPU; FP4 weights have no \"\n                \"Hopper (SM90) kernel. Use an FP8 checkpoint, or run on a Blackwell GPU.\"\n            )\n        return\n\n    # SM100: DeepGEMM has no float32 scale-factor path.\n    if scale.dtype == torch.float32:\n        raise NotImplementedError(\n            \"DeepGEMM has no float32 scale-factor path on Blackwell (SM100): these scales are plain float32 \"\n            \"(quantization_config.scale_fmt='float'), and rounding them to UE8M0 would silently corrupt the \"\n            \"output. Use a checkpoint quantized with scale_fmt='ue8m0', or a path that consumes float32 block \"\n            \"scales directly — the FP8 linear falls back to Triton automatically; for experts use \"\n            \"`model.set_experts_implementation('grouped_mm')`.\"\n        )\n\n\ndef _ceil_to_ue8m0(sf: torch.Tensor) -> torch.Tensor:\n    \"\"\"Round each fp32 SF up to the nearest power of 2 (zero mantissa).","sourceCodeStart":312,"sourceCodeEnd":348,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/integrations/deepgemm.py#L312-L348","documentation":"Error \"DeepGEMM's FP4 (int8-packed) path requires a Blackwell (SM100+) GPU; FP4 weights have no Hopper (SM90) kernel. Use an FP8 checkpoint, or run on a Blackwell GPU.\" thrown in huggingface/transformers.","triggerScenarios":"Raised in DeepGEMM FP4 path when the GPU is not Blackwell (SM100+) since FP4 kernels only exist there.","commonSituations":"Loading an FP4 (int8-packed) DeepGEMM checkpoint on Hopper or older GPUs.","solutions":["Use an FP8 checkpoint instead of FP4 on Hopper GPUs.","Run on a Blackwell (SM100+) GPU for the FP4 path."],"exampleFix":null,"handlingStrategy":null,"validationCode":null,"typeGuard":null,"tryCatchPattern":null,"preventionTips":[],"tags":[],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}