sgl-project/sglang · error · RuntimeError
is not supported on XPU (no XPU kernel implementation).
Error message
{scheme.__class__.__name__} is not supported on XPU (no XPU kernel implementation). What it means
Raised by compressed-tensors quantization when a config whose quant scheme has no XPU kernel implementation is applied on the XPU device. The scheme class name is included in the message; the request fails during weight loading.
Solutions
- Use an FP8 W8A8 quantized checkpoint on XPU
- Use an unquantized model (non-quantized layers fall back to UnquantizedLinearMethod)
- Run on CUDA/NPU hardware where the scheme is supported
Defensive patterns
Strategy: validation
Validate before calling
import torch
_is_xpu = hasattr(torch, "xpu") and torch.xpu.is_available()
cfg = model_cfg["quantization_config"]["config"]
is_fp8 = cfg.get("weights", {}).get("type") == "float-8" or "fp8" in str(cfg).lower()
if _is_xpu and not is_fp8:
raise SystemExit("XPU supports only W8A8-FP8 compressed-tensors schemes") Prevention
- Use FP8 W8A8 checkpoints on Intel GPUs
- Gate CI on device type when validating quantized checkpoints
- Run unquantized models on XPU if FP8 quantization is unavailable
When it happens
Trigger: Running sglang with an XPU device (torch.xpu available) and a compressed-tensors checkpoint whose linear scheme resolves to anything other than CompressedTensorsW8A8Fp8, e.g. W8A8-Int8 or W4A16 quantized model.
Common situations: Serving compressed-tensor-quantized checkpoints on Intel Arc/Flex/PVC GPUs where only the FP8 kernel path is implemented; using NVIDIA-oriented quantized models on XPU.
Related errors
- Ascend A2/A3 NPU does not support nvfp4…
- Comfy full_precision_matrix_mult does not support fused…
- Comfy NVFP4 layer needs U8 packed weights and FP8 block…
- ComfyFp8Config must be constructed from safetensors layer…
- For FP8 Fused MoE layer, we require either per tensor or…
AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28).
Data as JSON: /api/errors/08903b8ee16b1647.
Report an issue: GitHub.
Appendix: source
Thrown at python/sglang/srt/layers/quantization/compressed_tensors/compressed_tensors.py:982
"not supported by Compressed Tensors. "
"Falling back to UnquantizedLinearMethod"
)
return None
else:
# Find the quant_scheme
scheme = self._get_scheme_from_parts( # type: ignore
weight_quant=weight_quant,
input_quant=input_quant,
format=scheme_format,
)
# Raise error if device does not support the scheme
# (e.g. fp8 needs ada lovelace)
# Note: NPU devices do not support min_capability function
if _is_xpu:
if not isinstance(scheme, CompressedTensorsW8A8Fp8):
raise RuntimeError(
f"{scheme.__class__.__name__} is not supported on XPU "
"(no XPU kernel implementation)."
)
elif not _is_npu:
self._check_scheme_supported(scheme.get_min_capability())
logger.debug("Using scheme: %s for %s", scheme.__class__.__name__, layer_name)
return scheme
def get_lm_head_scheme(
self, layer: torch.nn.Module, layer_name: Optional[str] = None
) -> Optional[CompressedTensorsLinearScheme]:
"""Resolve the scheme for a ParallelLMHead, or None if the checkpoint
stores the head unquantized.
The head is treated as quantized only when a config target names it by
layer name (exact or ``re:`` regex, e.g. ``re:.*lm_head``). Module-type
targets like ``Linear`` are not consulted: llm-compressor emits those
for decoder linears, and checkpoints following the common conventionView on GitHub (pinned to 0132848349)