{"record":{"id":"5165eb83a20e18df","repo":"vllm-project/vllm","slug":"the-optimized-moe-wna16-gemm-kernel-is-only-availa","errorCode":null,"errorMessage":"The optimized moe_wna16_gemm kernel is only available on CUDA platforms","messagePattern":"The optimized moe_wna16_gemm kernel is only available on CUDA platforms","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"vllm/_custom_ops.py","lineNumber":2341,"sourceCode":"\ndef moe_wna16_gemm(\n    input: torch.Tensor,\n    output: torch.Tensor,\n    b_qweight: torch.Tensor,\n    b_scales: torch.Tensor,\n    b_qzeros: torch.Tensor | None,\n    topk_weights: torch.Tensor | None,\n    sorted_token_ids: torch.Tensor,\n    experts_ids: torch.Tensor,\n    num_tokens_post_pad: torch.Tensor,\n    top_k: int,\n    BLOCK_SIZE_M: int,\n    BLOCK_SIZE_N: int,\n    BLOCK_SIZE_K: int,\n    bit: int,\n) -> torch.Tensor:\n    if not current_platform.is_cuda():\n        raise NotImplementedError(\n            \"The optimized moe_wna16_gemm kernel is only available on CUDA platforms\"\n        )\n    torch.ops._moe_C.moe_wna16_gemm(\n        input,\n        output,\n        b_qweight,\n        b_scales,\n        b_qzeros,\n        topk_weights,\n        sorted_token_ids,\n        experts_ids,\n        num_tokens_post_pad,\n        top_k,\n        BLOCK_SIZE_M,\n        BLOCK_SIZE_N,\n        BLOCK_SIZE_K,\n        bit,\n    )","sourceCodeStart":2323,"sourceCodeEnd":2359,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/_custom_ops.py#L2323-L2359","documentation":"moe_wna16_gemm dispatches directly to torch.ops._moe_C.moe_wna16_gemm, a compiled CUDA extension for weight-only (WnA16, e.g. W4A16/W8A16) grouped MoE GEMM. There is no CPU/XPU/ROCm implementation, so the Python wrapper raises NotImplementedError before touching the extension when current_platform.is_cuda() is False.","triggerScenarios":"Calling vllm._custom_ops.moe_wna16_gemm(...) on a non-CUDA platform, or in an environment where the CUDA platform detector does not identify as CUDA (CPU-only build, XPU, etc.).","commonSituations":"Unit-testing MoE kernel wrappers on a CPU machine; running an int4/int8 weight-only MoE model (e.g. Qwen/Mixtral GPTQ variants) with VLLM_PLATFORM set to a non-CUDA backend.","solutions":["Run on a CUDA GPU with a CUDA-enabled vLLM build","Switch to a quantization format whose MoE path has kernels for your platform (e.g. unquantized or a marlin-compatible format)","In tests, skip or mock this wrapper when torch.cuda.is_available() is False"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"from vllm.platforms import current_platform\nassert current_platform.is_cuda(), \"moe_wna16_gemm requires CUDA\"","typeGuard":"def moe_wna16_available() -> bool:\n    from vllm.platforms import current_platform\n    return current_platform.is_cuda()","tryCatchPattern":"if not moe_wna16_available():\n    out = torch_reference_moe_gemm(...)  # fallback impl\nelse:\n    out = ops.moe_wna16_gemm(...)","preventionTips":["Skip MoE kernel tests on non-CUDA CI runners","Check current_platform.is_cuda() before dispatching to CUDA-only MoE wrappers"],"tags":["moe","quantization","weight-only","cuda-only","platform-support"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}