sgl-project/sglang · error · ValueError

expert-pack v1 supports only single-GPU TP=EP=1

Error message

expert-pack v1 supports only single-GPU TP=EP=1

What it means

expert-pack v1 weight layout only supports single-GPU execution: both moe_ep_size and moe_tp_size must be 1. Any tensor-parallel or expert-parallel degree > 1 cannot shard the packed expert store and is rejected in create_weights.

Source

Thrown at python/sglang/srt/layers/quantization/expert_pack.py:104

        self.situ_linear_beta: float | None = None

    def create_weights(
        self,
        layer: torch.nn.Module,
        num_experts: int,
        hidden_size: int,
        intermediate_size_per_partition: int,
        params_dtype: torch.dtype,
        **extra_weight_attrs,
    ) -> None:
        del extra_weight_attrs
        if layer.num_fused_shared_experts:
            raise ValueError(
                "expert-pack requires --disable-shared-experts-fusion so the "
                "shared expert remains on the dense GGUF path"
            )
        if layer.moe_ep_size != 1 or layer.moe_tp_size != 1:
            raise ValueError("expert-pack v1 supports only single-GPU TP=EP=1")
        if num_experts != self.store.header.num_experts:
            raise ValueError("FusedMoE expert count does not match expert-pack")
        if params_dtype not in (torch.bfloat16, torch.float16):
            raise ValueError("expert-pack kernel requires BF16 or FP16 activations")
        gate_shape = self.store.entries[(layer.layer_id, 0, 0)].shape
        down_shape = self.store.entries[(layer.layer_id, 0, 2)].shape
        if gate_shape != (hidden_size, intermediate_size_per_partition):
            raise ValueError(
                f"expert-pack gate shape {gate_shape} does not match "
                f"{(hidden_size, intermediate_size_per_partition)}"
            )
        if down_shape != (intermediate_size_per_partition, hidden_size):
            raise ValueError(
                f"expert-pack down shape {down_shape} does not match "
                f"{(intermediate_size_per_partition, hidden_size)}"
            )
        self.layer_id = layer.layer_id
        self.hidden_size = hidden_size

View on GitHub (pinned to 0132848349)

Solutions

  1. Run with --tp 1 --ep-size 1 (single GPU)
  2. If multi-GPU is required, use a non-expert-pack quantization of the model
  3. Watch for the companion errors: expert count mismatch and dtype checks in the same function

Example fix

# before
python -m sglang.launch_server --model expert-pack-model --tp 8
# after
python -m sglang.launch_server --model expert-pack-model --tp 1
Defensive patterns

Strategy: validation

Validate before calling

assert tp_size == 1 and ep_size == 1, "expert-pack v1 supports only TP=EP=1"

Prevention

When it happens

Trigger: Launching an expert-pack model with --tp N (N>1) or --ep-size > 1, so layer.moe_ep_size or layer.moe_tp_size exceeds 1.

Common situations: Trying to speed up an expert-pack model by sharding across GPUs; default multi-GPU launch scripts.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/4384154b135e9632. Report an issue: GitHub.