{"record":{"id":"7a123d15c1dd715e","repo":"vllm-project/vllm","slug":"the-qutlass-c-extension-is-not-loaded-make-sur","errorCode":null,"errorMessage":"The `_qutlass_C` extension is not loaded. Make sure your custom op library is imported before calling fusedQuantizeMx.","messagePattern":"The `_qutlass_C` extension is not loaded\\. Make sure your custom op library is imported before calling fusedQuantizeMx\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"vllm/_custom_ops.py","lineNumber":4117,"sourceCode":"    if b.device != a.device:\n        raise ValueError(\"`a` and `b` must be on the same device.\")\n\n    xh_e2m1 = torch.empty(\n        *a.shape[:-1], a.size(-1) // 2, dtype=torch.uint8, device=a.device\n    )\n\n    rows, cols = a.numel() // a.size(-1), a.size(-1) // 32\n    n_row_blocks = cdiv(rows, 128)\n    n_col_blocks = cdiv(cols, 4)\n    padded_rows = n_row_blocks * 128\n    padded_cols = n_col_blocks * 4\n\n    xh_e8m0 = torch.empty(\n        padded_rows, padded_cols, dtype=torch.float8_e8m0fnu, device=a.device\n    )\n\n    if not hasattr(torch.ops, \"_qutlass_C\"):\n        raise RuntimeError(\n            \"The `_qutlass_C` extension is not loaded. \"\n            \"Make sure your custom op library is imported before calling fusedQuantizeMx.\"\n        )\n\n    if method == \"quest\":\n        return torch.ops._qutlass_C.fusedQuantizeMxQuest(a, b, xh_e2m1, xh_e8m0)\n    elif method == \"abs_max\":\n        return torch.ops._qutlass_C.fusedQuantizeMxAbsMax(a, b, xh_e2m1, xh_e8m0)\n    else:\n        raise ValueError(f\"invalid method {method!r}, must be 'quest' or 'abs_max'\")\n\n\nif hasattr(torch.ops._qutlass_C, \"fusedQuantizeNvAbsMax\"):\n\n    @register_fake(\"_qutlass_C::fusedQuantizeNvAbsMax\")\n    def _fake_fused_quantize_nv_absmax(\n        a: torch.Tensor,\n        b: torch.Tensor,","sourceCodeStart":4099,"sourceCodeEnd":4135,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/_custom_ops.py#L4099-L4135","documentation":"fusedQuantizeMx() dispatches to torch.ops._qutlass_C.fusedQuantizeMxQuest / fusedQuantizeMxAbsMax. That namespace is only registered when the CUTLASS-based _qutlass_C extension shared library has been imported/loaded; if it is not, the wrapper raises RuntimeError instead of letting torch.ops raise an opaque AttributeError.","triggerScenarios":"Calling vllm._custom_ops.fusedQuantizeMx() in an environment where the _qutlass_C extension was never loaded — CPU-only vLLM wheel, a stripped build, or calling before the module that registers the ops (vllm import chain) has run.","commonSituations":"Unit-testing the wrapper on a machine without the compiled extension; importing _custom_ops directly in a minimal script without importing vllm first; a broken/partial wheel install where the .so is missing.","solutions":["import vllm (or the module that loads custom op libraries, e.g. vllm._quatlss_C loader) before calling the op","Verify the extension loads: check hasattr(torch.ops, '_qutlass_C') after importing vllm; reinstall vLLM if absent on a CUDA build","Skip the code path on platforms where the extension is not built"],"exampleFix":"# before\nfrom vllm import _custom_ops as ops\nq, s = ops.fusedQuantizeMx(a, b)  # _qutlass_C never loaded\n# after\nimport vllm  # loads custom op libraries first\nfrom vllm import _custom_ops as ops\nassert hasattr(torch.ops, \"_qutlass_C\")\nq, s = ops.fusedQuantizeMx(a, b)","handlingStrategy":"type-guard","validationCode":"import torch, vllm\nif not hasattr(torch.ops, \"_qutlass_C\"):\n    raise RuntimeError(\"build lacks _qutlass_C; use a CUDA vLLM build\")","typeGuard":"def fused_quantize_mx_available() -> bool:\n    import torch, vllm  # vllm import loads op libraries\n    return hasattr(torch.ops, \"_qutlass_C\")","tryCatchPattern":"try:\n    q, s = ops.fusedQuantizeMx(a, b, method=\"abs_max\")\nexcept RuntimeError as e:\n    if \"_qutlass_C\" in str(e):\n        q, s = torch_ref_mx_quant(a)  # reference path\n    else:\n        raise","preventionTips":["Import vllm before touching _custom_ops in scripts","Gate exotic quant paths on hasattr(torch.ops, '_qutlass_C')"],"tags":["quantization","extension-loading","cuda","environment"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}