{"record":{"id":"84f79cda43c1de1a","repo":"huggingface/transformers","slug":"deepgemm-mega-moe-requires-fp4-packed-expert-weigh","errorCode":null,"errorMessage":"DeepGEMM Mega MoE requires FP4-packed expert weights (dtype=`int8`), got `{self.gate_up_proj.dtype}`. Use the 'deepgemm' dispatch for FP8 experts.","messagePattern":"DeepGEMM Mega MoE requires FP4-packed expert weights \\(dtype=`int8`\\), got `(.+?)`\\. Use the 'deepgemm' dispatch for FP8 experts\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"src/transformers/integrations/deepgemm.py","lineNumber":869,"sourceCode":"\n    `process_group` is supplied automatically by `MoeTensorParalellExperts._prepare_input_fn`\n    when the module is wrapped for TP — it's required for the symm-buffer rendezvous\n    on first forward. `top_k_index` is GLOBAL expert ids (`-1` marks skipped slots).\n\n    Caller-managed `self` attributes:\n      - `gate_up_proj`, `gate_up_proj_scale_inv`: L1 weight + UE8M0 SF.\n      - `down_proj`, `down_proj_scale_inv`: L2 weight + UE8M0 SF.\n      Both pairs must be transformed together via\n      `transform_weights_for_mega_moe((gate_up, gate_up_sf), (down, down_sf))`.\n      - `config.swiglu_limit` (optional): SwiGLU clamp; absent → unclamped.\n    \"\"\"\n    # Fail before the (hub-download + JIT) load if this device can't serve these dtypes. Mega MoE is\n    # Blackwell-only, and its weights are always FP4 (int8) — so the FP4 arch check doubles as the\n    # SM100 gate; the explicit `!= int8` check below covers a non-FP4 (misconfigured) checkpoint.\n    _assert_sm100_requirements(self.gate_up_proj, self.down_proj_scale_inv)\n\n    if self.gate_up_proj.dtype != torch.int8:\n        raise NotImplementedError(\n            f\"DeepGEMM Mega MoE requires FP4-packed expert weights (dtype=`int8`), got \"\n            f\"`{self.gate_up_proj.dtype}`. Use the 'deepgemm' dispatch for FP8 experts.\"\n        )\n\n    if process_group is None:\n        raise ValueError(\n            \"DeepGEMM Mega MoE requires a `process_group` for the EP group. The TP wrapping \"\n            \"(MoeTensorParalellMegaMoeExperts) supplies it automatically; pass it explicitly otherwise.\"\n        )\n\n    deepgemm = load_deepgemm_kernel()\n\n    # First-forward one-shot: pack UE8M0 SFs and interleave the L1/L2 weights for UTCCP.\n    # Kept lazy here (instead of in a quantizer load-time hook) so the megamoe-specific\n    # setup lives alongside the megamoe forward — `set_experts_implementation` refuses\n    # to flip in/out of `deepgemm_megamoe` at runtime, so the flag won't go stale.\n    if not getattr(self, \"_megamoe_transformed\", False):\n        setup_megamoe_weights(self)","sourceCodeStart":851,"sourceCodeEnd":887,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/integrations/deepgemm.py#L851-L887","documentation":"Error \"DeepGEMM Mega MoE requires FP4-packed expert weights (dtype=`int8`), got `{self.gate_up_proj.dtype}`. Use the 'deepgemm' dispatch for FP8 experts.\" thrown in huggingface/transformers.","triggerScenarios":"Raised in DeepGEMM Mega MoE when expert weights are not FP4-packed int8.","commonSituations":"Using Mega MoE dispatch with FP8 expert weights; FP8 experts must use the regular deepgemm dispatch.","solutions":["Use FP4-packed (int8 dtype) expert weights with Mega MoE.","Use the 'deepgemm' dispatch for FP8 experts instead."],"exampleFix":null,"handlingStrategy":null,"validationCode":null,"typeGuard":null,"tryCatchPattern":null,"preventionTips":[],"tags":[],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}