{"record":{"id":"6d8c3fd0d2685ca2","repo":"sgl-project/sglang","slug":"quantization-nvfp4-online-is-supported-only-on-n","errorCode":null,"errorMessage":"--quantization nvfp4_online is supported only on NVIDIA Blackwell SM100/SM103 GPUs.","messagePattern":"--quantization nvfp4_online is supported only on NVIDIA Blackwell SM100/SM103 GPUs\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/arg_groups/overrides.py","lineNumber":2739,"sourceCode":"            f\"dp_size, got tp_size={view.tp_size}, dp_size={view.dp_size}.\"\n        )\n        assert view.attn_cp_size == 1, (\n            \"--enable-tp-lm-head-all-to-all currently requires \"\n            f\"attn_cp_size == 1, got {view.attn_cp_size}.\"\n        )\n    return {}\n\n\n@register_post_process\ndef _moe_runner_backend_quant_constraints(view: Any) -> dict:\n    \"\"\"The quantization-driven moe_runner_backend resolutions at the head of\n    _handle_moe_kernel_config. The backend-compatibility asserts and the\n    disable_shared_experts_fusion writes (post-publish writers exist for that\n    field) stay in the handler.\"\"\"\n    moe_runner_backend = view.moe_runner_backend\n    if view.quantization == \"nvfp4_online\":\n        if not is_sm100_supported():\n            raise ValueError(\n                \"--quantization nvfp4_online is supported only on \"\n                \"NVIDIA Blackwell SM100/SM103 GPUs.\"\n            )\n        if moe_runner_backend == \"auto\":\n            moe_runner_backend = \"flashinfer_trtllm\"\n        elif moe_runner_backend not in [\n            \"flashinfer_trtllm\",\n            \"flashinfer_trtllm_routed\",\n            \"flashinfer_cutedsl\",\n        ]:\n            raise ValueError(\n                \"--quantization nvfp4_online supports only \"\n                \"--moe-runner-backend flashinfer_trtllm or \"\n                \"flashinfer_trtllm_routed, or flashinfer_cutedsl.\"\n            )\n    # Ascend runs MXFP8 MoE on the Ascend runner; every backend selected below is\n    # CUDA/ROCm-only. Forcing one here would not merely pick the wrong runner:\n    # FusedMoE keys its w1/w3 shard swap (\"flashinfer assumes w31\") and its","sourceCodeStart":2721,"sourceCodeEnd":2757,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/arg_groups/overrides.py#L2721-L2757","documentation":"Online NVFP4 quantization (--quantization nvfp4_online) relies on Blackwell SM100/SM103 hardware instructions (FP4 tensor cores and block-scaled conversion), so SGLang rejects it at startup on older architectures.","triggerScenarios":"Launching with --quantization nvfp4_online on a GPU that is not SM100/SM103 (H100, A100, AMD, SM12x consumer Blackwell).","commonSituations":"Trying NVFP4 online quantization on an H100 dev machine or in CI without Blackwell runners.","solutions":["Run on an SM100/SM103 Blackwell GPU (B200/GB200/B300)","Use a quantization supported by your hardware (e.g. fp8 on Hopper, or a pre-quantized checkpoint with a matching scheme)"],"exampleFix":"# before (on H100)\n--quantization nvfp4_online\n# after\n--quantization fp8_dynauto_float","handlingStrategy":"validation","validationCode":"import torch\nif args.quantization == \"nvfp4_online\":\n    assert torch.cuda.get_device_capability() in ((10, 0), (10, 3)), \"nvfp4_online needs SM100/SM103\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Map quantization schemes to minimum hardware in your deployment matrix"],"tags":["sglang","nvfp4","quantization","hardware-gpu","blackwell"],"backgroundTag":"gpu-architecture-not-supported","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}