huggingface/transformers · error · ValueError

DeepGEMM Mega MoE requires a `process_group` for the EP grou

Error message

DeepGEMM Mega MoE requires a `process_group` for the EP group. The TP wrapping (MoeTensorParalellMegaMoeExperts) supplies it automatically; pass it explicitly otherwise.

What it means

Error "DeepGEMM Mega MoE requires a `process_group` for the EP group. The TP wrapping (MoeTensorParalellMegaMoeExperts) supplies it automatically; pass it explicitly otherwise." thrown in huggingface/transformers.

Source

Thrown at src/transformers/integrations/deepgemm.py:875

      - `gate_up_proj`, `gate_up_proj_scale_inv`: L1 weight + UE8M0 SF.
      - `down_proj`, `down_proj_scale_inv`: L2 weight + UE8M0 SF.
      Both pairs must be transformed together via
      `transform_weights_for_mega_moe((gate_up, gate_up_sf), (down, down_sf))`.
      - `config.swiglu_limit` (optional): SwiGLU clamp; absent → unclamped.
    """
    # Fail before the (hub-download + JIT) load if this device can't serve these dtypes. Mega MoE is
    # Blackwell-only, and its weights are always FP4 (int8) — so the FP4 arch check doubles as the
    # SM100 gate; the explicit `!= int8` check below covers a non-FP4 (misconfigured) checkpoint.
    _assert_sm100_requirements(self.gate_up_proj, self.down_proj_scale_inv)

    if self.gate_up_proj.dtype != torch.int8:
        raise NotImplementedError(
            f"DeepGEMM Mega MoE requires FP4-packed expert weights (dtype=`int8`), got "
            f"`{self.gate_up_proj.dtype}`. Use the 'deepgemm' dispatch for FP8 experts."
        )

    if process_group is None:
        raise ValueError(
            "DeepGEMM Mega MoE requires a `process_group` for the EP group. The TP wrapping "
            "(MoeTensorParalellMegaMoeExperts) supplies it automatically; pass it explicitly otherwise."
        )

    deepgemm = load_deepgemm_kernel()

    # First-forward one-shot: pack UE8M0 SFs and interleave the L1/L2 weights for UTCCP.
    # Kept lazy here (instead of in a quantizer load-time hook) so the megamoe-specific
    # setup lives alongside the megamoe forward — `set_experts_implementation` refuses
    # to flip in/out of `deepgemm_megamoe` at runtime, so the flag won't go stale.
    if not getattr(self, "_megamoe_transformed", False):
        setup_megamoe_weights(self)
        self._megamoe_transformed = True

    num_top_k = top_k_index.size(-1)
    num_tokens = hidden_states.size(0)
    hidden_dim = hidden_states.size(-1)
    num_local_experts = self.gate_up_proj.size(0)

View on GitHub (pinned to a597f97485)

Solutions

1. Pass the EP `process_group` explicitly when constructing Mega MoE experts outside the TP wrapper.
2. Use MoeTensorParalellMegaMoeExperts which supplies the group automatically.

When it happens

Trigger: Raised in DeepGEMM Mega MoE when no expert-parallel process_group is available.

Common situations: Instantiating Mega MoE experts outside the TP/EP wrapping that normally supplies the EP process group.

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/85b750227637f579. Report an issue: GitHub.