{"record":{"id":"bbdbb46882b1a899","repo":"sgl-project/sglang","slug":"current-platform-does-not-support-nvfp4-quantizati","errorCode":null,"errorMessage":"Current platform does not support NVFP4 quantization. Please use Blackwell and above.","messagePattern":"Current platform does not support NVFP4 quantization\\. Please use Blackwell and above\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/compressed_tensors/schemes/compressed_tensors_w4a4_nvfp4_moe.py","lineNumber":37,"sourceCode":")\nfrom sglang.srt.utils import set_weight_attrs\n\nlogger = logging.getLogger(__name__)\n\n__all__ = [\"CompressedTensorsW4A4Nvfp4MoE\"]\n\nif TYPE_CHECKING:\n    from sglang.srt.layers.moe.token_dispatcher import (\n        CombineInput,\n        StandardDispatchOutput,\n    )\n\n\nclass CompressedTensorsW4A4Nvfp4MoE(CompressedTensorsMoEScheme):\n\n    def __init__(self):\n        if not is_blackwell_supported():\n            raise ValueError(\n                \"Current platform does not support NVFP4\"\n                \" quantization. Please use Blackwell and\"\n                \" above.\"\n            )\n        self.group_size = 16\n        self.use_flashinfer_trtllm = get_moe_runner_backend().is_flashinfer_trtllm()\n\n    @property\n    def load_up_proj_weight_first(self) -> bool:\n        \"\"\"Load W13 as ``[up; gate]`` for CUTLASS; TRT-LLM reorders post-load.\"\"\"\n        return not self.use_flashinfer_trtllm\n\n    @classmethod\n    def get_min_capability(cls) -> int:\n        # Requires sm100(blackwell) architecture\n        return 100\n\n    def create_weights(","sourceCodeStart":19,"sourceCodeEnd":55,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/compressed_tensors/schemes/compressed_tensors_w4a4_nvfp4_moe.py#L19-L55","documentation":"The NVFP4 W4A4 MoE scheme is selected but the current GPU is not Blackwell or newer. NVFP4 fused MoE kernels rely on native FP4 tensor-core instructions only present on SM100+, so the scheme refuses to initialize on older hardware.","triggerScenarios":"Loading a checkpoint quantized to NVFP4 (e.g. NVFP4 DeepSeek/LLaMA variants) on H100/A100/Ada or any pre-Blackwell GPU; is_blackwell_supported() returns False in CompressedTensorsW4A4Nvfp4MoE.__init__.","commonSituations":"Running an NVFP4 model on a dev box with older GPUs; CI runners without Blackwell; cluster node selection landing on the wrong node type.","solutions":["Run on a Blackwell (B200/GB200/RTX 50-series) or newer GPU","Re-quantize the model to FP8 or INT8 which are supported on the available hardware","If on a Blackwell card but still failing, update CUDA driver and PyTorch/flashinfer so compute capability is detected as sm100+"],"exampleFix":"# before: python -m sglang.launch_server --model nvfp4-model --tp 8   # on H100\n# after: run on B200 node, or use the fp8 checkpoint:\npython -m sglang.launch_server --model fp8-model --tp 8","handlingStrategy":"validation","validationCode":"import sglang.srt.utils as u\n# before loading an NVFP4 checkpoint:\nfrom sglang.srt.layers.quantization.utils import is_blackwell_supported\nif not is_blackwell_supported(): raise SystemExit(\"need Blackwell GPU for NVFP4\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Gate NVFP4 deployments on a GPU capability check (sm >= 100)","Keep an FP8 fallback checkpoint for non-Blackwell nodes"],"tags":["nvfp4","gpu-hardware","blackwell","quantization","moe"],"backgroundTag":"gpu-architecture-not-supported","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}