sgl-project/sglang · error · RuntimeError

Unsupported interleaved_rope_fp64 dtype: {dtype}

Error message

Unsupported interleaved_rope_fp64 dtype: {dtype}

What it means

The JIT-compiled interleaved RoPE fp64 kernel is specialized for bfloat16 only; the template instantiation for other dtypes is not built. Passing float16 or float32 triggers this RuntimeError at module compile time.

Source

Thrown at python/sglang/kernels/ops/diffusion/rope/interleaved_rope_fp64_jit.py:17

from __future__ import annotations

from typing import TYPE_CHECKING

import torch

from sglang.kernels.jit.utils import cache_once, load_jit, make_cpp_args
from sglang.srt.utils.custom_op import register_custom_op

if TYPE_CHECKING:
    from tvm_ffi.module import Module


@cache_once
def _jit_interleaved_rope_fp64_module(dtype: torch.dtype) -> Module:
    if dtype is not torch.bfloat16:
        raise RuntimeError(f"Unsupported interleaved_rope_fp64 dtype: {dtype}")
    args = make_cpp_args(dtype)
    return load_jit(
        "diffusion_interleaved_rope_fp64",
        *args,
        cuda_files=["diffusion/interleaved_rope_fp64.cuh"],
        cuda_wrappers=[
            (
                "interleaved_rope_fp64",
                f"interleaved_rope_fp64::InterleavedRopeFP64Kernel<{args}>::run",
            ),
        ],
    )


def _fake_impl(
    q: torch.Tensor,
    k: torch.Tensor,
    cos: torch.Tensor,

View on GitHub (pinned to 0132848349)

Solutions

1. Cast Q/K to torch.bfloat16 before calling fused_interleaved_rope_fp64.
2. Change the model's dtype config to bfloat16.
3. Use a fallback RoPE implementation if another dtype is required.

Example fix

// before
q = q.to(torch.float16)
out = fused_interleaved_rope_fp64(q, k, cos, sin)
// after
q = q.to(torch.bfloat16)
k = k.to(torch.bfloat16)
out = fused_interleaved_rope_fp64(q, k, cos, sin)

if q.dtype is not torch.bfloat16:
    q, k = q.bfloat16(), k.bfloat16()

def is_bf16(t: torch.Tensor) -> bool:
    return t.dtype is torch.bfloat16

When it happens

Trigger: Calling fused_interleaved_rope_fp64 with a fp16/fp32 Q or K tensor; the dtype check happens before load_jit.

Common situations: Running a diffusion model variant configured for fp16 instead of bf16, or casting inputs to fp32 for debugging.

Related errors

• Unsupported ltx25_decoder_rope dtype: {dtype}
• QKV tensors must be CUDA bfloat16 tensors
• Unsupported usp_merge_heads dtype: {dtype}
• timestep must be a CUDA bfloat16 tensor
• Unsupported modulate_scale_shift dtype: {dtype}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/c5413f5468cf83ea. Report an issue: GitHub.