sgl-project/sglang · error · AssertionError

kernel dispatch requires at least one tensor argument

Error message

kernel dispatch requires at least one tensor argument

What it means

_inputs_on_cuda scans (*args, **kwargs) for torch.Tensor arguments to decide whether to launch the fused Triton kernel or the CPU reference implementation. If no argument is a tensor at all, dispatch is impossible and it raises AssertionError — this is a programming error in the caller, not a runtime condition.

Source

Thrown at python/sglang/kernels/ops/attention/dsv4_attn_metadata_kernels.py:19

from __future__ import annotations

from typing import Optional

import msgspec
import torch
import triton
import triton.language as tl


def _inputs_on_cuda(*args, **kwargs) -> bool:
    """Route kernel dispatch by input placement: the first tensor argument
    decides. CUDA inputs take the fused triton kernel; CPU inputs take the
    torch reference implementation (triton is CUDA-only, and CPU-side callers
    such as unit tests exercise the reference path)."""
    for value in (*args, *kwargs.values()):
        if isinstance(value, torch.Tensor):
            return value.is_cuda
    raise AssertionError("kernel dispatch requires at least one tensor argument")


class ExpandPrefillCausallyResult(msgspec.Struct):
    seq_lens_casual: torch.Tensor
    req_pool_indices_repeated: torch.Tensor


class ExpandPrefillCausally:
    @classmethod
    def execute(cls, *args, **kwargs) -> ExpandPrefillCausallyResult:
        if _inputs_on_cuda(*args, **kwargs):
            return cls.triton(*args, **kwargs)
        return cls.torch(*args, **kwargs)

    @classmethod
    def torch(
        cls,
        *,

View on GitHub (pinned to 0132848349)

Solutions

1. Pass at least one torch.Tensor argument so device dispatch can work
2. If arguments are nested (dataclass, list), unpack them before calling the op
3. Fix the call site — this assert indicates an API misuse bug, not something to catch

assert any(isinstance(a, torch.Tensor) for a in (*args, *kwargs.values())), \
    "op requires at least one tensor argument"

When it happens

Trigger: Invoking the wrapped kernel-op (via execute) with only scalars/None and zero tensor arguments, e.g. a degenerate test call or a refactor that dropped the tensor arguments.

Common situations: Unit tests calling the op with placeholder args; a refactor that moved tensors into a struct the wrapper does not unpack; passing tensors inside a dataclass/list instead of as direct args.

Related errors

• Missing previous frame for delta payload
• {self._op_label()}: no triton backend
• sparse_attn_v4_paged_decode expects fp16/bf16 q, got {q.dtyp
• kv_scales supplied but unified_kv is {unified_kv.dtype}, exp
• unified_kv dtype mismatch: kv={unified_kv.dtype}, q={q.dtype

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/f84aabc194090d55. Report an issue: GitHub.