{"record":{"id":"900ab15000b0c35e","repo":"sgl-project/sglang","slug":"can-t-import-trtllm-fp8-block-scale-routed-moe-fro","errorCode":null,"errorMessage":"Can't import trtllm_fp8_block_scale_routed_moe from flashinfer. Please check flashinfer version.","messagePattern":"Can't import trtllm_fp8_block_scale_routed_moe from flashinfer\\. Please check flashinfer version\\.","errorType":"exception","errorClass":"ImportError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/moe/flashinfer_trtllm_moe.py","lineNumber":189,"sourceCode":"    n_group: Optional[int],\n    topk_group: Optional[int],\n    intermediate_size: int,\n    local_expert_offset: int,\n    local_num_experts: int,\n    routed_scaling_factor: Optional[float],\n    output: torch.Tensor,\n    routing_method_type: int = 0,\n    use_shuffled_weight: bool = False,\n    weight_layout: int = 0,\n    enable_pdl: Optional[bool] = None,\n    tune_max_num_tokens: int = 8192,\n    fp8_quantization_type: Optional[int] = None,\n    activation_type: Optional[int] = None,\n) -> None:\n    try:\n        from flashinfer.fused_moe import trtllm_fp8_block_scale_routed_moe\n    except ImportError as e:\n        raise ImportError(\n            \"Can't import trtllm_fp8_block_scale_routed_moe from flashinfer. \"\n            \"Please check flashinfer version.\"\n        ) from e\n\n    kwargs = {\n        \"topk_ids\": topk_ids,\n        \"routing_bias\": routing_bias,\n        \"hidden_states\": hidden_states,\n        \"hidden_states_scale\": hidden_states_scale,\n        \"gemm1_weights\": gemm1_weights,\n        \"gemm1_weights_scale\": gemm1_weights_scale,\n        \"gemm1_alpha\": gemm1_alpha,\n        \"gemm1_beta\": gemm1_beta,\n        \"gemm1_clamp_limit\": gemm1_clamp_limit,\n        \"gemm2_weights\": gemm2_weights,\n        \"gemm2_weights_scale\": gemm2_weights_scale,\n        \"output\": output,\n        \"num_experts\": num_experts,","sourceCodeStart":171,"sourceCodeEnd":207,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/moe/flashinfer_trtllm_moe.py#L171-L207","documentation":"Lazy import of trtllm_fp8_block_scale_routed_moe from flashinfer.fused_moe failed. This routed variant of the TRT-LLM fp8 block-scale MoE kernel likewise requires a recent flashinfer version; older releases don't export it.","triggerScenarios":"Calling trtllm_fp8_block_scale_routed_moe_out_wrapper (routed path of fused_experts_none_to_flashinfer_trtllm_fp8) on a flashinfer build without the symbol.","commonSituations":"Same as the non-routed variant: stale flashinfer pin, broken compiled extension import, or a nightly-vs-stable API difference.","solutions":["Upgrade flashinfer to a version exporting trtllm_fp8_block_scale_routed_moe","Test the import directly: python -c \"from flashinfer.fused_moe import trtllm_fp8_block_scale_routed_moe\"","Reinstall flashinfer matched to your torch/CUDA build if the import fails due to extension loading","Disable the routed trtllm path or use the triton fallback backend"],"exampleFix":"# before\npip install flashinfer-python==0.1.*\n# after\npip install -U flashinfer-python\npython -c \"from flashinfer.fused_moe import trtllm_fp8_block_scale_routed_moe\"","handlingStrategy":"fallback","validationCode":"try:\n    from flashinfer.fused_moe import trtllm_fp8_block_scale_routed_moe  # noqa\n    HAS_ROUTED = True\nexcept ImportError:\n    HAS_ROUTED = False\nif not HAS_ROUTED:\n    raise RuntimeError('flashinfer lacks routed trtllm fp8 MoE; upgrade flashinfer')","typeGuard":null,"tryCatchPattern":"try:\n    trtllm_fp8_block_scale_routed_moe_out_wrapper(...)\nexcept ImportError as e:\n    if 'trtllm_fp8_block_scale_routed_moe' in str(e):\n        disable_routed_path_or_use_triton()\n    else:\n        raise","preventionTips":["Upgrade flashinfer together with sglang upgrades","Verify new flashinfer symbols in CI before enabling the routed path"],"tags":["flashinfer","trtllm","moe","fp8","import-error","version-mismatch"],"backgroundTag":"missing-optional-dependency","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}