vllm-project/vllm · error · NotImplementedError

asymmetric int8 activation quantization is unsupported on XP

Error message

asymmetric int8 activation quantization is unsupported on XPU

What it means

scaled_int8_quant() has no compiled _C kernel on Intel XPU, so it falls back to a torch reference implementation that only supports symmetric quantization. Requesting asymmetric int8 quantization (symmetric=False, which needs azp) on XPU therefore raises NotImplementedError.

Source

Thrown at vllm/_custom_ops.py:2009

) -> tuple[torch.Tensor, torch.Tensor, torch.Tensor | None]:
    """
    Quantize the input tensor to int8 and return the quantized tensor and scale, and maybe azp.

    Args:
        input: The input tensor to be quantized to int8.
        scale: Optional scaling factor for the int8 quantization.
            When not provided, we invoke dynamic-per-token quantization.
        azp: Optional zero-point for the int8 quantization.
            Must be provided for asymmetric quantization if `scale` is provided.
        symmetric: Whether to use symmetric quantization (scale only, azp ignored).

    Returns:
      tuple[torch.Tensor, torch.Tensor, torch.Tensor | None] : Output int8 tensor, scales, and optionally azp.
    """
    if current_platform.is_xpu():
        # XPU has no _C int8 quant op; use the torch.compile reference.
        if not symmetric:
            raise NotImplementedError(
                "asymmetric int8 activation quantization is unsupported on XPU"
            )
        if scale is not None:
            q = (input.to(torch.float32) / scale).round().clamp(-128, 127)
            return q.to(torch.int8), scale, None

        from vllm._xpu_ops import xpu_ops

        q, scales, _ = xpu_ops.dynamic_per_token_int8_quant_ref(
            input.contiguous(), True, 8
        )
        return q, scales.reshape(-1, 1).to(torch.float32), None

    output = torch.empty_like(input, dtype=torch.int8)
    if scale is not None:
        # static-per-tensor quantization.
        assert symmetric == (azp is None), (
            "azp must only be provided for asymmetric quantization."

View on GitHub (pinned to c794754062)

Solutions

Use a symmetrically-quantized checkpoint (weight-only or symmetric W8A8) on XPU
Pass symmetric=True and drop azp if the model tolerates symmetric activation quantization
Run the workload on CUDA/ROCm hardware where the fused _C int8 kernel supports asymmetric mode

Example fix

# before
q, s, a = ops.scaled_int8_quant(x, scale, azp, symmetric=False)  # on XPU -> raises
# after
q, s, a = ops.scaled_int8_quant(x, scale, symmetric=True)  # XPU-supported path

Defensive patterns

Strategy: type-guard

Validate before calling

from vllm.platforms import current_platform
if current_platform.is_xpu():
    assert symmetric, "asymmetric int8 quant unsupported on XPU; use symmetric=True"

Type guard

def int8_quant_supported(symmetric: bool) -> bool:
    return symmetric or not current_platform.is_cuda() is False and not current_platform.is_xpu()

Try / catch

try:
    q, s, a = ops.scaled_int8_quant(x, scale, azp, symmetric=symmetric)
except NotImplementedError:
    q, s, a = ops.scaled_int8_quant(x, scale, symmetric=True)  # XPU fallback

Prevention

Choose symmetric-quantized checkpoints for XPU deployments
Gate quantization config on current_platform in multi-platform code

When it happens

Trigger: Calling vllm._custom_ops.scaled_int8_quant(input, scale, azp, symmetric=False) on a platform where current_platform.is_xpu() is True; typically with a static scale + azp pair from a quantized checkpoint.

Common situations: Serving a W8A8 asymmetric-quantized model (e.g. some AWQ/GPTQ int8 or compressed-tensors int8 checkpoints) on an Intel GPU (Intel Data Center / Arc with VLLM_PLATFORM=xpu); works on CUDA, fails on XPU.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/c8ec47823220245b. Report an issue: GitHub.