sgl-project/sglang · error · NotImplementedError

Unsupported runner backend: %s

Error message

Unsupported runner backend: %s

What it means

Fp8MoEMethod.apply dispatches on the runner backend (deep_gemm, flashinfer_*, triton, hpc_ops, etc.); if runner.runner_backend matches none of the recognized branches, it raises NotImplementedError as a defensive catch-all. Seeing it means a backend enum value exists that the dispatcher wasn't updated to handle.

Source

Thrown at python/sglang/srt/layers/quantization/fp8.py:2623

                ),
                output1_scales_gate_scalar=(
                    getattr(layer, "output1_scales_gate_scalar", None)
                    if not self.block_quant
                    else None
                ),
                output2_scales_scalar=(
                    getattr(layer, "output2_scales_scalar", None)
                    if not self.block_quant
                    else None
                ),
                activation_type=activation_type,
            )
        elif self.runner.runner_backend.is_hpc_ops():
            quant_info = self._get_hpc_ops_quant_info(layer)
        elif self.runner.runner_backend.is_triton():
            quant_info = self.get_triton_quant_info(layer)
        else:
            raise NotImplementedError(
                "Unsupported runner backend: %s" % self.runner.runner_backend
            )

        return self.runner.run(dispatch_output, quant_info)

    def _ensure_cutlass_buffers_initialized(self, layer: Module) -> None:
        if getattr(self, "_cutlass_buffers_ready", False):
            return

        device = layer.w13_weight.device
        num_experts = layer.w13_weight.shape[0]
        hidden_size = layer.w2_weight.shape[1]
        intermediate_size_per_partition = layer.intermediate_size_per_partition
        w13_num_shards = 2 if layer.moe_runner_config.is_gated else 1

        self.ab_strides1 = torch.full(
            (num_experts,), hidden_size, device=device, dtype=torch.int64
        )

View on GitHub (pinned to 0132848349)

Solutions

1. Update SGLang so the dispatcher in fp8.py handles your runner backend (add an elif branch)
2. Use a standard --moe-runner-backend value (auto, triton, deep_gemm, flashinfer_trtllm, hpc_ops)
3. Reinstall/align sglang versions so enums and dispatch code come from the same release

Example fix

# before
--moe-runner-backend my_custom_backend
# after
--moe-runner-backend auto

try:
    out = method.apply(layer, dispatch_output)
except NotImplementedError as e:
    logger.error("runner backend %s unsupported by this fp8 dispatcher; falling back", method.runner.runner_backend)
    raise SystemExit("use a standard --moe-runner-backend") from e

When it happens

Trigger: Calling the FP8 MoE apply path with a new/renamed MoeRunnerBackend enum value not covered by the if/elif chain — usually after a partial upgrade, custom backend additions, or a mismatched enum import.

Common situations: Custom forks adding a runner backend without updating fp8.py dispatch; version skew between sglang packages; passing a raw string where an enum is expected.

Related errors

• expert-pack index role or rank is invalid
• Kimi expert-pack {role} quant type is unsupported
• The hpc_ops MoE runner backend only supports FP8-quantized M
• Unknown gemm type: {gemm_type}
• fuse_swiglu_interleaved set on an incompatible fused_moe cal

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/f3d713ced42a981b. Report an issue: GitHub.