sgl-project/sglang · error · ValueError

Current platform does not support NVFP4 quantization. Please

Error message

Current platform does not support NVFP4 quantization. Please use Blackwell and above.

What it means

The NVFP4 W4A4 MoE scheme is selected but the current GPU is not Blackwell or newer. NVFP4 fused MoE kernels rely on native FP4 tensor-core instructions only present on SM100+, so the scheme refuses to initialize on older hardware.

Source

Thrown at python/sglang/srt/layers/quantization/compressed_tensors/schemes/compressed_tensors_w4a4_nvfp4_moe.py:37

)
from sglang.srt.utils import set_weight_attrs

logger = logging.getLogger(__name__)

__all__ = ["CompressedTensorsW4A4Nvfp4MoE"]

if TYPE_CHECKING:
    from sglang.srt.layers.moe.token_dispatcher import (
        CombineInput,
        StandardDispatchOutput,
    )


class CompressedTensorsW4A4Nvfp4MoE(CompressedTensorsMoEScheme):

    def __init__(self):
        if not is_blackwell_supported():
            raise ValueError(
                "Current platform does not support NVFP4"
                " quantization. Please use Blackwell and"
                " above."
            )
        self.group_size = 16
        self.use_flashinfer_trtllm = get_moe_runner_backend().is_flashinfer_trtllm()

    @property
    def load_up_proj_weight_first(self) -> bool:
        """Load W13 as ``[up; gate]`` for CUTLASS; TRT-LLM reorders post-load."""
        return not self.use_flashinfer_trtllm

    @classmethod
    def get_min_capability(cls) -> int:
        # Requires sm100(blackwell) architecture
        return 100

    def create_weights(

View on GitHub (pinned to 0132848349)

Solutions

1. Run on a Blackwell (B200/GB200/RTX 50-series) or newer GPU
2. Re-quantize the model to FP8 or INT8 which are supported on the available hardware
3. If on a Blackwell card but still failing, update CUDA driver and PyTorch/flashinfer so compute capability is detected as sm100+

Example fix

# before: python -m sglang.launch_server --model nvfp4-model --tp 8   # on H100
# after: run on B200 node, or use the fp8 checkpoint:
python -m sglang.launch_server --model fp8-model --tp 8

import sglang.srt.utils as u
# before loading an NVFP4 checkpoint:
from sglang.srt.layers.quantization.utils import is_blackwell_supported
if not is_blackwell_supported(): raise SystemExit("need Blackwell GPU for NVFP4")

When it happens

Trigger: Loading a checkpoint quantized to NVFP4 (e.g. NVFP4 DeepSeek/LLaMA variants) on H100/A100/Ada or any pre-Blackwell GPU; is_blackwell_supported() returns False in CompressedTensorsW4A4Nvfp4MoE.__init__.

Common situations: Running an NVFP4 model on a dev box with older GPUs; CI runners without Blackwell; cluster node selection landing on the wrong node type.

Related errors

• --quantization nvfp4_online is supported only on NVIDIA Blac
• InklingNvfp4MoEMethod is the dense shared-expert method; rou
• Type must match: {self.a_dtype} != {self.b_dtype}
• nvfp4_gemm_swiglu_nvfp4_quant currently supports NVFP4 input
• nvfp4_gemm_swiglu_nvfp4_quant requires SM100, got SM{major}{

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/bbdbb46882b1a899. Report an issue: GitHub.