sgl-project/sglang · error · RuntimeError

is not supported on XPU (no XPU kernel implementation).

Error message

{scheme.__class__.__name__} is not supported on XPU (no XPU kernel implementation).

What it means

Raised by compressed-tensors quantization when a config whose quant scheme has no XPU kernel implementation is applied on the XPU device. The scheme class name is included in the message; the request fails during weight loading.

Solutions

  1. Use an FP8 W8A8 quantized checkpoint on XPU
  2. Use an unquantized model (non-quantized layers fall back to UnquantizedLinearMethod)
  3. Run on CUDA/NPU hardware where the scheme is supported
Defensive patterns

Strategy: validation

Validate before calling

import torch
_is_xpu = hasattr(torch, "xpu") and torch.xpu.is_available()
cfg = model_cfg["quantization_config"]["config"]
is_fp8 = cfg.get("weights", {}).get("type") == "float-8" or "fp8" in str(cfg).lower()
if _is_xpu and not is_fp8:
    raise SystemExit("XPU supports only W8A8-FP8 compressed-tensors schemes")

Prevention

When it happens

Trigger: Running sglang with an XPU device (torch.xpu available) and a compressed-tensors checkpoint whose linear scheme resolves to anything other than CompressedTensorsW8A8Fp8, e.g. W8A8-Int8 or W4A16 quantized model.

Common situations: Serving compressed-tensor-quantized checkpoints on Intel Arc/Flex/PVC GPUs where only the FP8 kernel path is implemented; using NVIDIA-oriented quantized models on XPU.

Related errors


AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/08903b8ee16b1647. Report an issue: GitHub.

Appendix: source

Thrown at python/sglang/srt/layers/quantization/compressed_tensors/compressed_tensors.py:982

                "not supported by Compressed Tensors. "
                "Falling back to UnquantizedLinearMethod"
            )
            return None

        else:
            # Find the quant_scheme
            scheme = self._get_scheme_from_parts(  # type: ignore
                weight_quant=weight_quant,
                input_quant=input_quant,
                format=scheme_format,
            )

        # Raise error if device does not support the scheme
        # (e.g. fp8 needs ada lovelace)
        # Note: NPU devices do not support min_capability function
        if _is_xpu:
            if not isinstance(scheme, CompressedTensorsW8A8Fp8):
                raise RuntimeError(
                    f"{scheme.__class__.__name__} is not supported on XPU "
                    "(no XPU kernel implementation)."
                )
        elif not _is_npu:
            self._check_scheme_supported(scheme.get_min_capability())
        logger.debug("Using scheme: %s for %s", scheme.__class__.__name__, layer_name)
        return scheme

    def get_lm_head_scheme(
        self, layer: torch.nn.Module, layer_name: Optional[str] = None
    ) -> Optional[CompressedTensorsLinearScheme]:
        """Resolve the scheme for a ParallelLMHead, or None if the checkpoint
        stores the head unquantized.

        The head is treated as quantized only when a config target names it by
        layer name (exact or ``re:`` regex, e.g. ``re:.*lm_head``). Module-type
        targets like ``Linear`` are not consulted: llm-compressor emits those
        for decoder linears, and checkpoints following the common convention

View on GitHub (pinned to 0132848349)