sgl-project/sglang · critical · ValueError

num_fused_shared_experts > 1 ({self.num_fused_shared_experts

Error message

num_fused_shared_experts > 1 ({self.num_fused_shared_experts}) is not supported on CUDA platform. The current TopK implementation only handles one fused shared expert. AMD platform with aiter supports multiple shared experts.

What it means

For Bailing v3, fusing shared experts into the TopK output is only implemented for one shared expert on CUDA; grouped_topk_gpu/_post_process_topk_ids only handle the last ids column. Multiple fused shared experts only work on AMD with the aiter kernel.

Source

Thrown at python/sglang/srt/models/bailing_moe_v3.py:1456

                        "--disable-shared-experts-fusion to use padding solution instead."
                    )
        return None

    def determine_num_fused_shared_experts(self):
        self.num_fused_shared_experts = (
            0
            if is_shared_experts_fusion_disabled()
            else getattr(self.config, "num_shared_experts", 0)
        )
        if self.num_fused_shared_experts == 0:
            return

        # Safety check: current CUDA implementation only supports num_fused_shared_experts == 1.
        # The grouped_topk_gpu and _post_process_topk_ids functions only handle the last column,
        # which is incorrect when num_fused_shared_experts > 1.
        # AMD platform with aiter handles this correctly via fused_append_shared_experts kernel.
        if self.num_fused_shared_experts > 1 and not _is_hip:
            raise ValueError(
                f"num_fused_shared_experts > 1 ({self.num_fused_shared_experts}) is not "
                f"supported on CUDA platform. The current TopK implementation only handles "
                f"one fused shared expert. AMD platform with aiter supports multiple shared experts."
            )

        moe_ep_size = get_parallel().moe_ep_size
        if moe_ep_size > 1:
            log_info_on_rank0(
                logger,
                f"Shared experts fusion optimization is enabled with {self.num_fused_shared_experts} fused shared expert(s) under EP mode (ep_size={moe_ep_size}). "
                f"Shared experts will be distributed across GPUs along with routed experts.",
            )
        else:
            log_info_on_rank0(
                logger,
                f"Shared experts fusion optimization is enabled with {self.num_fused_shared_experts} fused shared expert(s).",
            )

View on GitHub (pinned to 0132848349)

Solutions

1. Set num_fused_shared_experts to 1 (or 0) in config.json / disable shared-expert fusion on CUDA
2. Run on AMD with aiter if multiple fused shared experts are required
3. Upgrade SGLang in case multi-fused support was added for CUDA later

Example fix

// before
"num_fused_shared_experts": 2
// after
"num_fused_shared_experts": 1

import torch
assert getattr(config, "num_fused_shared_experts", 1) <= 1 or torch.version.hip

When it happens

Trigger: Loading a Bailing v3 config with num_fused_shared_experts > 1 on an NVIDIA GPU (not HIP).

Common situations: Taking a config tuned for AMD/aiter and running it on CUDA; models with 2+ shared experts enabled for fusion.

Related errors

• num_token_non_padded and x must be on the same device
• topk_ids must be a CUDA tensor
• The hpc_ops MoE runner backend does not support fused shared
• expert-pack requires --disable-shared-experts-fusion so the
• Inkling shared-sink LoRA outer factors must have expert dime

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/f71591566cdea777. Report an issue: GitHub.