sgl-project/sglang · error · RuntimeError

{scheme.class.name} is not supported on XPU (no XPU

Error message

{scheme.__class__.__name__} is not supported on XPU (no XPU kernel implementation).

What it means

On XPU (Intel GPU) devices, only CompressedTensorsW8A8Fp8 has kernel implementations; any other resolved linear scheme (INT8, W4A16, W8A16, etc.) raises RuntimeError naming the scheme class.

Source

Thrown at python/sglang/srt/layers/quantization/compressed_tensors/compressed_tensors.py:982

                "not supported by Compressed Tensors. "
                "Falling back to UnquantizedLinearMethod"
            )
            return None

        else:
            # Find the quant_scheme
            scheme = self._get_scheme_from_parts(  # type: ignore
                weight_quant=weight_quant,
                input_quant=input_quant,
                format=scheme_format,
            )

        # Raise error if device does not support the scheme
        # (e.g. fp8 needs ada lovelace)
        # Note: NPU devices do not support min_capability function
        if _is_xpu:
            if not isinstance(scheme, CompressedTensorsW8A8Fp8):
                raise RuntimeError(
                    f"{scheme.__class__.__name__} is not supported on XPU "
                    "(no XPU kernel implementation)."
                )
        elif not _is_npu:
            self._check_scheme_supported(scheme.get_min_capability())
        logger.debug("Using scheme: %s for %s", scheme.__class__.__name__, layer_name)
        return scheme

    def get_lm_head_scheme(
        self, layer: torch.nn.Module, layer_name: Optional[str] = None
    ) -> Optional[CompressedTensorsLinearScheme]:
        """Resolve the scheme for a ParallelLMHead, or None if the checkpoint
        stores the head unquantized.

        The head is treated as quantized only when a config target names it by
        layer name (exact or ``re:`` regex, e.g. ``re:.*lm_head``). Module-type
        targets like ``Linear`` are not consulted: llm-compressor emits those
        for decoder linears, and checkpoints following the common convention

View on GitHub (pinned to 0132848349)

Solutions

Use an FP8 W8A8 quantized checkpoint on XPU
Use an unquantized model (non-quantized layers fall back to UnquantizedLinearMethod)
Run on CUDA/NPU hardware where the scheme is supported

Defensive patterns

Strategy: validation

Validate before calling

import torch
_is_xpu = hasattr(torch, "xpu") and torch.xpu.is_available()
cfg = model_cfg["quantization_config"]["config"]
is_fp8 = cfg.get("weights", {}).get("type") == "float-8" or "fp8" in str(cfg).lower()
if _is_xpu and not is_fp8:
    raise SystemExit("XPU supports only W8A8-FP8 compressed-tensors schemes")

Prevention

Use FP8 W8A8 checkpoints on Intel GPUs
Gate CI on device type when validating quantized checkpoints
Run unquantized models on XPU if FP8 quantization is unavailable

When it happens

Trigger: Running sglang with an XPU device (torch.xpu available) and a compressed-tensors checkpoint whose linear scheme resolves to anything other than CompressedTensorsW8A8Fp8, e.g. W8A8-Int8 or W4A16 quantized model.

Common situations: Serving compressed-tensor-quantized checkpoints on Intel Arc/Flex/PVC GPUs where only the FP8 kernel path is implemented; using NVIDIA-oriented quantized models on XPU.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/08903b8ee16b1647. Report an issue: GitHub.