sgl-project/sglang · error · RuntimeError

Cannot restore flashinfer TRT-LLM BF16 MoE weight shape for

Error message

Cannot restore flashinfer TRT-LLM BF16 MoE weight shape for {weight_name}: current shape={tuple(param.data.shape)}, expected shape={expected_shape}.

What it means

Error "Cannot restore flashinfer TRT-LLM BF16 MoE weight shape for {weight_name}: current shape={tuple(param.data.shape)}, expected shape={expected_shape}." thrown in sgl-project/sglang.

Source

Thrown at python/sglang/srt/layers/quantization/unquant.py:695

            w13_rows = (
                2 * layer.intermediate_size_per_partition
                if layer.moe_runner_config.is_gated
                else layer.intermediate_size_per_partition
            )
            expected_shape = (layer.num_local_experts, w13_rows, layer.hidden_size)
        elif weight_name.endswith(".experts.w2_weight"):
            expected_shape = (
                layer.num_local_experts,
                layer.hidden_size,
                layer.intermediate_size_per_partition,
            )

        if expected_shape is None or tuple(param.data.shape) == expected_shape:
            return

        expected_numel = expected_shape[0] * expected_shape[1] * expected_shape[2]
        if param.data.numel() != expected_numel:
            raise RuntimeError(
                f"Cannot restore flashinfer TRT-LLM BF16 MoE weight shape for {weight_name}: "
                f"current shape={tuple(param.data.shape)}, expected shape={expected_shape}."
            )

        param.data = param.data.reshape(expected_shape)

    def _aiter_ck_moe_supported(self, layer) -> bool:
        # aiter CK fused-MoE requires intermediate_size_per_partition to be 128-aligned
        # (GemmSpec=Default; otherwise CK raises "not support this GEMM problem").
        return layer.intermediate_size_per_partition % 128 == 0

    def create_moe_runner(
        self, layer: torch.nn.Module, moe_runner_config: MoeRunnerConfig
    ):
        self.moe_runner_config = moe_runner_config
        if self.use_flashinfer_trtllm_moe:
            backend = (
                MoeRunnerBackend.FLASHINFER_TRTLLM_ROUTED

View on GitHub (pinned to 0132848349)

When it happens

Trigger: Thrown at python/sglang/srt/layers/quantization/unquant.py:695 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/56cd89cf3083aa60. Report an issue: GitHub.