sgl-project/sglang · error · ImportError

Can't import trtllm_fp8_block_scale_routed_moe from flashinf

Error message

Can't import trtllm_fp8_block_scale_routed_moe from flashinfer. Please check flashinfer version.

What it means

Lazy import of trtllm_fp8_block_scale_routed_moe from flashinfer.fused_moe failed. This routed variant of the TRT-LLM fp8 block-scale MoE kernel likewise requires a recent flashinfer version; older releases don't export it.

Source

Thrown at python/sglang/srt/layers/moe/flashinfer_trtllm_moe.py:189

    n_group: Optional[int],
    topk_group: Optional[int],
    intermediate_size: int,
    local_expert_offset: int,
    local_num_experts: int,
    routed_scaling_factor: Optional[float],
    output: torch.Tensor,
    routing_method_type: int = 0,
    use_shuffled_weight: bool = False,
    weight_layout: int = 0,
    enable_pdl: Optional[bool] = None,
    tune_max_num_tokens: int = 8192,
    fp8_quantization_type: Optional[int] = None,
    activation_type: Optional[int] = None,
) -> None:
    try:
        from flashinfer.fused_moe import trtllm_fp8_block_scale_routed_moe
    except ImportError as e:
        raise ImportError(
            "Can't import trtllm_fp8_block_scale_routed_moe from flashinfer. "
            "Please check flashinfer version."
        ) from e

    kwargs = {
        "topk_ids": topk_ids,
        "routing_bias": routing_bias,
        "hidden_states": hidden_states,
        "hidden_states_scale": hidden_states_scale,
        "gemm1_weights": gemm1_weights,
        "gemm1_weights_scale": gemm1_weights_scale,
        "gemm1_alpha": gemm1_alpha,
        "gemm1_beta": gemm1_beta,
        "gemm1_clamp_limit": gemm1_clamp_limit,
        "gemm2_weights": gemm2_weights,
        "gemm2_weights_scale": gemm2_weights_scale,
        "output": output,
        "num_experts": num_experts,

View on GitHub (pinned to 0132848349)

Solutions

1. Upgrade flashinfer to a version exporting trtllm_fp8_block_scale_routed_moe
2. Test the import directly: python -c "from flashinfer.fused_moe import trtllm_fp8_block_scale_routed_moe"
3. Reinstall flashinfer matched to your torch/CUDA build if the import fails due to extension loading
4. Disable the routed trtllm path or use the triton fallback backend

Example fix

# before
pip install flashinfer-python==0.1.*
# after
pip install -U flashinfer-python
python -c "from flashinfer.fused_moe import trtllm_fp8_block_scale_routed_moe"

try:
    from flashinfer.fused_moe import trtllm_fp8_block_scale_routed_moe  # noqa
    HAS_ROUTED = True
except ImportError:
    HAS_ROUTED = False
if not HAS_ROUTED:
    raise RuntimeError('flashinfer lacks routed trtllm fp8 MoE; upgrade flashinfer')

try:
    trtllm_fp8_block_scale_routed_moe_out_wrapper(...)
except ImportError as e:
    if 'trtllm_fp8_block_scale_routed_moe' in str(e):
        disable_routed_path_or_use_triton()
    else:
        raise

When it happens

Trigger: Calling trtllm_fp8_block_scale_routed_moe_out_wrapper (routed path of fused_experts_none_to_flashinfer_trtllm_fp8) on a flashinfer build without the symbol.

Common situations: Same as the non-routed variant: stale flashinfer pin, broken compiled extension import, or a nightly-vs-stable API difference.

Related errors

• Can't import trtllm_fp8_block_scale_moe from flashinfer. Ple
• Kimi-K3 DCP with decode_attention_backend='cutedsl_mla' requ
• Failed to import 'set_transfer_engine' from 'mooncake.pg'. P
• Unsupported cute dtype {input.dtype}
• CuteDSL masked MoE supports activation 'silu' (gated) or 're

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/900ab15000b0c35e. Report an issue: GitHub.