{"record":{"id":"c8ec47823220245b","repo":"vllm-project/vllm","slug":"asymmetric-int8-activation-quantization-is-unsuppo","errorCode":null,"errorMessage":"asymmetric int8 activation quantization is unsupported on XPU","messagePattern":"asymmetric int8 activation quantization is unsupported on XPU","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"vllm/_custom_ops.py","lineNumber":2009,"sourceCode":") -> tuple[torch.Tensor, torch.Tensor, torch.Tensor | None]:\n    \"\"\"\n    Quantize the input tensor to int8 and return the quantized tensor and scale, and maybe azp.\n\n    Args:\n        input: The input tensor to be quantized to int8.\n        scale: Optional scaling factor for the int8 quantization.\n            When not provided, we invoke dynamic-per-token quantization.\n        azp: Optional zero-point for the int8 quantization.\n            Must be provided for asymmetric quantization if `scale` is provided.\n        symmetric: Whether to use symmetric quantization (scale only, azp ignored).\n\n    Returns:\n      tuple[torch.Tensor, torch.Tensor, torch.Tensor | None] : Output int8 tensor, scales, and optionally azp.\n    \"\"\"\n    if current_platform.is_xpu():\n        # XPU has no _C int8 quant op; use the torch.compile reference.\n        if not symmetric:\n            raise NotImplementedError(\n                \"asymmetric int8 activation quantization is unsupported on XPU\"\n            )\n        if scale is not None:\n            q = (input.to(torch.float32) / scale).round().clamp(-128, 127)\n            return q.to(torch.int8), scale, None\n\n        from vllm._xpu_ops import xpu_ops\n\n        q, scales, _ = xpu_ops.dynamic_per_token_int8_quant_ref(\n            input.contiguous(), True, 8\n        )\n        return q, scales.reshape(-1, 1).to(torch.float32), None\n\n    output = torch.empty_like(input, dtype=torch.int8)\n    if scale is not None:\n        # static-per-tensor quantization.\n        assert symmetric == (azp is None), (\n            \"azp must only be provided for asymmetric quantization.\"","sourceCodeStart":1991,"sourceCodeEnd":2027,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/_custom_ops.py#L1991-L2027","documentation":"scaled_int8_quant() has no compiled _C kernel on Intel XPU, so it falls back to a torch reference implementation that only supports symmetric quantization. Requesting asymmetric int8 quantization (symmetric=False, which needs azp) on XPU therefore raises NotImplementedError.","triggerScenarios":"Calling vllm._custom_ops.scaled_int8_quant(input, scale, azp, symmetric=False) on a platform where current_platform.is_xpu() is True; typically with a static scale + azp pair from a quantized checkpoint.","commonSituations":"Serving a W8A8 asymmetric-quantized model (e.g. some AWQ/GPTQ int8 or compressed-tensors int8 checkpoints) on an Intel GPU (Intel Data Center / Arc with VLLM_PLATFORM=xpu); works on CUDA, fails on XPU.","solutions":["Use a symmetrically-quantized checkpoint (weight-only or symmetric W8A8) on XPU","Pass symmetric=True and drop azp if the model tolerates symmetric activation quantization","Run the workload on CUDA/ROCm hardware where the fused _C int8 kernel supports asymmetric mode"],"exampleFix":"# before\nq, s, a = ops.scaled_int8_quant(x, scale, azp, symmetric=False)  # on XPU -> raises\n# after\nq, s, a = ops.scaled_int8_quant(x, scale, symmetric=True)  # XPU-supported path","handlingStrategy":"type-guard","validationCode":"from vllm.platforms import current_platform\nif current_platform.is_xpu():\n    assert symmetric, \"asymmetric int8 quant unsupported on XPU; use symmetric=True\"","typeGuard":"def int8_quant_supported(symmetric: bool) -> bool:\n    return symmetric or not current_platform.is_cuda() is False and not current_platform.is_xpu()","tryCatchPattern":"try:\n    q, s, a = ops.scaled_int8_quant(x, scale, azp, symmetric=symmetric)\nexcept NotImplementedError:\n    q, s, a = ops.scaled_int8_quant(x, scale, symmetric=True)  # XPU fallback","preventionTips":["Choose symmetric-quantized checkpoints for XPU deployments","Gate quantization config on current_platform in multi-platform code"],"tags":["quantization","int8","xpu","intel-gpu","platform-support"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}