sgl-project/sglang · error · NotImplementedError

GptOssForCausalLM on Intel XPU only supports bfloat16 dtype,

Error message

GptOssForCausalLM on Intel XPU only supports bfloat16 dtype, but got '{cfg.dtype}'. Please use --dtype bfloat16 or remove --dtype to use auto.

What it means

On Intel XPU, GptOssForCausalLM in SGLang is only implemented for bfloat16 compute. dtype 'auto' is allowed (with a warning), but any other explicit --dtype (e.g. float16) raises NotImplementedError.

Source

Thrown at python/sglang/srt/arg_groups/overrides.py:1133

        elif is_xpu():
            overrides["attention_backend"] = "intel_xpu"
        elif is_hip():
            overrides["attention_backend"] = "aiter"
        elif not (is_mps() and use_mlx()):
            # Exempt MLX only -- it owns attention in its own runner.  macOS
            # without MLX still falls through to triton and fails fast below,
            # rather than landing on torch_native (no sliding window, no sinks).
            overrides["attention_backend"] = "triton"
    if is_xpu():
        # Check for bf16 dtype on Intel XPU. Reads the pristine dtype request,
        # which equals the legacy mid-branch read: dtype had no earlier writer
        # for this arch.
        if cfg.dtype == "auto":
            logger.warning(
                "GptOssForCausalLM on Intel XPU currently supports bfloat16 dtype only"
            )
        elif cfg.dtype not in ["bfloat16"]:
            raise NotImplementedError(
                f"GptOssForCausalLM on Intel XPU only supports bfloat16 dtype, "
                f"but got '{cfg.dtype}'. Please use --dtype bfloat16 or remove --dtype to use auto."
            )
    quantization_config = getattr(hf_config, "quantization_config", None)
    is_mxfp4_quant_format = (
        quantization_config is not None
        and quantization_config.get("quant_method") == "mxfp4"
    )
    if is_mxfp4_quant_format:
        # use bf16 for mxfp4 triton kernels
        overrides["dtype"] = "bfloat16"
    if cfg.moe_runner_backend == "auto":

        if is_sm100_supported() and is_mxfp4_quant_format:
            overrides["moe_runner_backend"] = "flashinfer_mxfp4"
            logger.warning(
                "Detected SM100 and MXFP4 quantization format for GPT-OSS model, enabling FlashInfer MXFP4 MOE kernel."
            )

View on GitHub (pinned to 0132848349)

Solutions

1. Remove --dtype (auto resolves to bfloat16) or set --dtype bfloat16
2. If you need fp8/fp4 quantization paths, check XPU support first — they are not enabled via --dtype here

Example fix

# before
--dtype float16
# after
--dtype bfloat16

if model_arch == 'GptOssForCausalLM' and device_type == 'xpu':
    server_args.dtype = 'bfloat16' if server_args.dtype in ('auto', None) else server_args.dtype
    assert server_args.dtype == 'bfloat16'

except NotImplementedError as e:
    if 'bfloat16' in str(e): server_args.dtype = 'bfloat16'; retry()
    raise

When it happens

Trigger: Launching gpt-oss on Intel GPU with --dtype float16 or float32; _gpt_oss_overrides sees cfg.dtype not in ['bfloat16'] and not 'auto'.

Common situations: Porting a CUDA launch script (fp16 for older GPUs) to an Intel XPU machine; defaulting to fp16 in an orchestrator template.

Related errors

• q must be torch.float8_e4m3fn, got {q.dtype}
• Unknown dtype: {sampling_params.dtype}
• Invalid dtype: {sampling_params.dtype}
• Unknown dtype: {dtype}
• Harmony does not support reasoning effort {reasoning_effort}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/fe190690f0c54c7a. Report an issue: GitHub.