{"record":{"id":"85b750227637f579","repo":"huggingface/transformers","slug":"deepgemm-mega-moe-requires-a-process-group-for-t","errorCode":null,"errorMessage":"DeepGEMM Mega MoE requires a `process_group` for the EP group. The TP wrapping (MoeTensorParalellMegaMoeExperts) supplies it automatically; pass it explicitly otherwise.","messagePattern":"DeepGEMM Mega MoE requires a `process_group` for the EP group\\. The TP wrapping \\(MoeTensorParalellMegaMoeExperts\\) supplies it automatically; pass it explicitly otherwise\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/transformers/integrations/deepgemm.py","lineNumber":875,"sourceCode":"      - `gate_up_proj`, `gate_up_proj_scale_inv`: L1 weight + UE8M0 SF.\n      - `down_proj`, `down_proj_scale_inv`: L2 weight + UE8M0 SF.\n      Both pairs must be transformed together via\n      `transform_weights_for_mega_moe((gate_up, gate_up_sf), (down, down_sf))`.\n      - `config.swiglu_limit` (optional): SwiGLU clamp; absent → unclamped.\n    \"\"\"\n    # Fail before the (hub-download + JIT) load if this device can't serve these dtypes. Mega MoE is\n    # Blackwell-only, and its weights are always FP4 (int8) — so the FP4 arch check doubles as the\n    # SM100 gate; the explicit `!= int8` check below covers a non-FP4 (misconfigured) checkpoint.\n    _assert_sm100_requirements(self.gate_up_proj, self.down_proj_scale_inv)\n\n    if self.gate_up_proj.dtype != torch.int8:\n        raise NotImplementedError(\n            f\"DeepGEMM Mega MoE requires FP4-packed expert weights (dtype=`int8`), got \"\n            f\"`{self.gate_up_proj.dtype}`. Use the 'deepgemm' dispatch for FP8 experts.\"\n        )\n\n    if process_group is None:\n        raise ValueError(\n            \"DeepGEMM Mega MoE requires a `process_group` for the EP group. The TP wrapping \"\n            \"(MoeTensorParalellMegaMoeExperts) supplies it automatically; pass it explicitly otherwise.\"\n        )\n\n    deepgemm = load_deepgemm_kernel()\n\n    # First-forward one-shot: pack UE8M0 SFs and interleave the L1/L2 weights for UTCCP.\n    # Kept lazy here (instead of in a quantizer load-time hook) so the megamoe-specific\n    # setup lives alongside the megamoe forward — `set_experts_implementation` refuses\n    # to flip in/out of `deepgemm_megamoe` at runtime, so the flag won't go stale.\n    if not getattr(self, \"_megamoe_transformed\", False):\n        setup_megamoe_weights(self)\n        self._megamoe_transformed = True\n\n    num_top_k = top_k_index.size(-1)\n    num_tokens = hidden_states.size(0)\n    hidden_dim = hidden_states.size(-1)\n    num_local_experts = self.gate_up_proj.size(0)","sourceCodeStart":857,"sourceCodeEnd":893,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/integrations/deepgemm.py#L857-L893","documentation":"Error \"DeepGEMM Mega MoE requires a `process_group` for the EP group. The TP wrapping (MoeTensorParalellMegaMoeExperts) supplies it automatically; pass it explicitly otherwise.\" thrown in huggingface/transformers.","triggerScenarios":"Raised in DeepGEMM Mega MoE when no expert-parallel process_group is available.","commonSituations":"Instantiating Mega MoE experts outside the TP/EP wrapping that normally supplies the EP process group.","solutions":["Pass the EP `process_group` explicitly when constructing Mega MoE experts outside the TP wrapper.","Use MoeTensorParalellMegaMoeExperts which supplies the group automatically."],"exampleFix":null,"handlingStrategy":null,"validationCode":null,"typeGuard":null,"tryCatchPattern":null,"preventionTips":[],"tags":[],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}