{"record":{"id":"4384154b135e9632","repo":"sgl-project/sglang","slug":"expert-pack-v1-supports-only-single-gpu-tp-ep-1","errorCode":null,"errorMessage":"expert-pack v1 supports only single-GPU TP=EP=1","messagePattern":"expert-pack v1 supports only single-GPU TP=EP=1","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/expert_pack.py","lineNumber":104,"sourceCode":"        self.situ_linear_beta: float | None = None\n\n    def create_weights(\n        self,\n        layer: torch.nn.Module,\n        num_experts: int,\n        hidden_size: int,\n        intermediate_size_per_partition: int,\n        params_dtype: torch.dtype,\n        **extra_weight_attrs,\n    ) -> None:\n        del extra_weight_attrs\n        if layer.num_fused_shared_experts:\n            raise ValueError(\n                \"expert-pack requires --disable-shared-experts-fusion so the \"\n                \"shared expert remains on the dense GGUF path\"\n            )\n        if layer.moe_ep_size != 1 or layer.moe_tp_size != 1:\n            raise ValueError(\"expert-pack v1 supports only single-GPU TP=EP=1\")\n        if num_experts != self.store.header.num_experts:\n            raise ValueError(\"FusedMoE expert count does not match expert-pack\")\n        if params_dtype not in (torch.bfloat16, torch.float16):\n            raise ValueError(\"expert-pack kernel requires BF16 or FP16 activations\")\n        gate_shape = self.store.entries[(layer.layer_id, 0, 0)].shape\n        down_shape = self.store.entries[(layer.layer_id, 0, 2)].shape\n        if gate_shape != (hidden_size, intermediate_size_per_partition):\n            raise ValueError(\n                f\"expert-pack gate shape {gate_shape} does not match \"\n                f\"{(hidden_size, intermediate_size_per_partition)}\"\n            )\n        if down_shape != (intermediate_size_per_partition, hidden_size):\n            raise ValueError(\n                f\"expert-pack down shape {down_shape} does not match \"\n                f\"{(intermediate_size_per_partition, hidden_size)}\"\n            )\n        self.layer_id = layer.layer_id\n        self.hidden_size = hidden_size","sourceCodeStart":86,"sourceCodeEnd":122,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/expert_pack.py#L86-L122","documentation":"expert-pack v1 weight layout only supports single-GPU execution: both moe_ep_size and moe_tp_size must be 1. Any tensor-parallel or expert-parallel degree > 1 cannot shard the packed expert store and is rejected in create_weights.","triggerScenarios":"Launching an expert-pack model with --tp N (N>1) or --ep-size > 1, so layer.moe_ep_size or layer.moe_tp_size exceeds 1.","commonSituations":"Trying to speed up an expert-pack model by sharding across GPUs; default multi-GPU launch scripts.","solutions":["Run with --tp 1 --ep-size 1 (single GPU)","If multi-GPU is required, use a non-expert-pack quantization of the model","Watch for the companion errors: expert count mismatch and dtype checks in the same function"],"exampleFix":"# before\npython -m sglang.launch_server --model expert-pack-model --tp 8\n# after\npython -m sglang.launch_server --model expert-pack-model --tp 1","handlingStrategy":"validation","validationCode":"assert tp_size == 1 and ep_size == 1, \"expert-pack v1 supports only TP=EP=1\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Reserve expert-pack models for single-GPU serving","Use standard quantization for multi-GPU deployments"],"tags":["expert-pack","moe","tensor-parallel","single-gpu"],"backgroundTag":"parallelism-unsupported-for-format","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}