sgl-project/sglang · error · ValueError

--enable-deepseek-v4-fp4-indexer requires SM100 or SM120 GPU

Error message

--enable-deepseek-v4-fp4-indexer requires SM100 or SM120 GPUs with DeepGEMM FP4 indexer support.

What it means

ServerArgs validation rejects --enable-deepseek-v4-fp4-indexer unless the GPU compute capability is SM100 (Blackwell B100/B200) or SM120 (Blackwell consumer). The DeepSeek V4 FP4 indexer path relies on DeepGEMM FP4 kernels that are compiled/enabled only for those architectures; on older SMs (Hopper, Ada, Ampere) it cannot run.

Source

Thrown at python/sglang/srt/server_args.py:9078

        if cfg.enable_deterministic_inference:
            envs.SGLANG_FLASHINFER_MOE_FUSED_FINALIZE.set("0")
        if cfg.debug_cuda_graph:
            if not (is_cuda() or is_hip()):
                logger.warning(
                    "--debug-cuda-graph is not supported on non CUDA/HIP devices. "
                    "Disabling breakable CUDA graph."
                )
                self._declare("_handle_environment_variables", debug_cuda_graph=False)
            else:
                envs.SGLANG_USE_BREAKABLE_CUDA_GRAPH.set("1")
                logger.warning(
                    "Debug mode for CUDA graph is enabled via breakable CUDA graph. "
                    "All operations will run eagerly through the graph capture/replay path."
                )
        if cfg.enable_deepseek_v4_fp4_indexer and not (
            is_sm100_supported() or is_sm120_supported()
        ):
            raise ValueError(
                "--enable-deepseek-v4-fp4-indexer requires SM100 or SM120 GPUs with "
                "DeepGEMM FP4 indexer support."
            )
        # FP8 W_o GEMM needs DeepGEMM JIT. Enable exactly where the runtime can run
        # it, mirroring the forward scale split: the ue8m0 path
        # (DEEPGEMM_SCALE_UE8M0, true sm100, default on) or an sm90 opt-in
        # fp32-scale path (use FP4 expert ckpt). Disable in every other case.
        if is_cuda() and envs.SGLANG_OPT_FP8_WO_A_GEMM.get():
            from sglang.srt.layers import deep_gemm_wrapper

            sm = get_device_sm()
            explicit = envs.SGLANG_OPT_FP8_WO_A_GEMM.is_set()
            supported = deep_gemm_wrapper.DEEPGEMM_SCALE_UE8M0 or (
                deep_gemm_wrapper.ENABLE_JIT_DEEPGEMM
                and is_sm90_supported()
                and explicit
            )
            if not supported and explicit:

View on GitHub (pinned to 0132848349)

Solutions

1. Remove --enable-deepseek-v4-fp4-indexer on non-Blackwell hardware and use the supported FP8/default indexer path
2. Run the model on an SM100/SM120 GPU (B200, GB200, RTX Blackwell) if FP4 indexer is required
3. Confirm the detected compute capability via torch.cuda.get_device_capability() to rule out device-masking mistakes

Example fix

# before (H100)
python -m sglang.launch_server --model deepseek-v4 --enable-deepseek-v4-fp4-indexer
# after (H100)
python -m sglang.launch_server --model deepseek-v4

def supports_fp4_indexer() -> bool:
    if not torch.cuda.is_available():
        return False
    major, _ = torch.cuda.get_device_capability(0)
    return major >= 100  # SM100/SM120 Blackwell

flags = ["--enable-deepseek-v4-fp4-indexer"] if supports_fp4_indexer() else []

When it happens

Trigger: Launching with --enable-deepseek-v4-fp4-indexer on a GPU whose compute capability is below SM100 (e.g. H100 sm90, A100 sm80, L40S sm89), where is_sm100_supported() and is_sm120_supported() both return False.

Common situations: Developing on H100/A100 clusters and reusing flags tuned for B200; mixing up FP8 (sm90-capable) and FP4 (sm100+) indexer requirements; drivers/runtime reporting unexpected compute capability in containers.

Related errors

• DeepSeekV4 only supports interleave CP strategy, got {cfg.cp
• DeepSeekV4 CP supports moe_a2a_backend in {supported_a2a_bac
• DSA indexer weights_proj LoRA is incompatible with piecewise
• DSA indexer only supports CUDA, HIP, and NPU
• DeepSeek-V4 FP4 experts require torch.float4_e2m1fn_x2 suppo

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/f01633ea0bb1415a. Report an issue: GitHub.