sgl-project/sglang · critical · ValueError
num_fused_shared_experts > 1
Error message
num_fused_shared_experts > 1 ({self.num_fused_shared_experts}) is not supported on CUDA platform. The current TopK implementation only handles one fused shared expert. AMD platform with aiter supports multiple shared experts. What it means
For Bailing v3, fusing shared experts into the TopK output is only implemented for one shared expert on CUDA; grouped_topk_gpu/_post_process_topk_ids only handle the last ids column. Multiple fused shared experts only work on AMD with the aiter kernel.
Solutions
- Set num_fused_shared_experts to 1 (or 0) in config.json / disable shared-expert fusion on CUDA
- Run on AMD with aiter if multiple fused shared experts are required
- Upgrade SGLang in case multi-fused support was added for CUDA later
Example fix
// before "num_fused_shared_experts": 2 // after "num_fused_shared_experts": 1
Defensive patterns
Strategy: validation
Validate before calling
import torch assert getattr(config, "num_fused_shared_experts", 1) <= 1 or torch.version.hip
Prevention
- On CUDA keep num_fused_shared_experts <= 1
- Reserve multi-shared-expert fusion for AMD+aiter
When it happens
Trigger: Loading a Bailing v3 config with num_fused_shared_experts > 1 on an NVIDIA GPU (not HIP).
Common situations: Taking a config tuned for AMD/aiter and running it on CUDA; models with 2+ shared experts enabled for fusion.
Related errors
- DeepEP v2 MoE has not validated fused shared experts yet…
- expert-pack requires --disable-shared-experts-fusion so the…
- Inkling shared-sink LoRA expert count does not match
- Inkling shared-sink LoRA outer factors must have expert…
- num_token_non_padded and x must be on the same device
AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28).
Data as JSON: /api/errors/f71591566cdea777.
Report an issue: GitHub.
Appendix: source
Thrown at python/sglang/srt/models/bailing_moe_v3.py:1456
"--disable-shared-experts-fusion to use padding solution instead."
)
return None
def determine_num_fused_shared_experts(self):
self.num_fused_shared_experts = (
0
if is_shared_experts_fusion_disabled()
else getattr(self.config, "num_shared_experts", 0)
)
if self.num_fused_shared_experts == 0:
return
# Safety check: current CUDA implementation only supports num_fused_shared_experts == 1.
# The grouped_topk_gpu and _post_process_topk_ids functions only handle the last column,
# which is incorrect when num_fused_shared_experts > 1.
# AMD platform with aiter handles this correctly via fused_append_shared_experts kernel.
if self.num_fused_shared_experts > 1 and not _is_hip:
raise ValueError(
f"num_fused_shared_experts > 1 ({self.num_fused_shared_experts}) is not "
f"supported on CUDA platform. The current TopK implementation only handles "
f"one fused shared expert. AMD platform with aiter supports multiple shared experts."
)
moe_ep_size = get_parallel().moe_ep_size
if moe_ep_size > 1:
log_info_on_rank0(
logger,
f"Shared experts fusion optimization is enabled with {self.num_fused_shared_experts} fused shared expert(s) under EP mode (ep_size={moe_ep_size}). "
f"Shared experts will be distributed across GPUs along with routed experts.",
)
else:
log_info_on_rank0(
logger,
f"Shared experts fusion optimization is enabled with {self.num_fused_shared_experts} fused shared expert(s).",
)
View on GitHub (pinned to 0132848349)