sgl-project/sglang · error · AssertionError
kernel dispatch requires at least one tensor argument
Error message
kernel dispatch requires at least one tensor argument
What it means
_inputs_on_cuda scans (*args, **kwargs) for torch.Tensor arguments to decide whether to launch the fused Triton kernel or the CPU reference implementation. If no argument is a tensor at all, dispatch is impossible and it raises AssertionError — this is a programming error in the caller, not a runtime condition.
Solutions
- Pass at least one torch.Tensor argument so device dispatch can work
- If arguments are nested (dataclass, list), unpack them before calling the op
- Fix the call site — this assert indicates an API misuse bug, not something to catch
Defensive patterns
Strategy: validation
Validate before calling
assert any(isinstance(a, torch.Tensor) for a in (*args, *kwargs.values())), \
"op requires at least one tensor argument" Prevention
- Unpack structured args (dataclasses/lists) before calling kernel ops
- This assert signals a call-site bug: fix it rather than catching it
When it happens
Trigger: Invoking the wrapped kernel-op (via execute) with only scalars/None and zero tensor arguments, e.g. a degenerate test call or a refactor that dropped the tensor arguments.
Common situations: Unit tests calling the op with placeholder args; a refactor that moved tensors into a struct the wrapper does not unpack; passing tensors inside a dataclass/list instead of as direct args.
Understand the failure class
Background: "must be a positive integer", "cannot be empty", "invalid argument": how invalid-argument errors work across open-source libraries — this error's family across 33 libraries.
Related errors
- appendExternalCorpusTokens called without…
- AttentionOffsetCache should not store data
- cannot mix lambda extractor with static kwargs
- combined_history=True requires direction=0 (bidi)
- comfy_nvfp4 is inferred from per-layer checkpoint metadata…
AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28).
Data as JSON: /api/errors/f84aabc194090d55.
Report an issue: GitHub.
Appendix: source
Thrown at python/sglang/kernels/ops/attention/dsv4_attn_metadata_kernels.py:19
from __future__ import annotations
from typing import Optional
import msgspec
import torch
import triton
import triton.language as tl
def _inputs_on_cuda(*args, **kwargs) -> bool:
"""Route kernel dispatch by input placement: the first tensor argument
decides. CUDA inputs take the fused triton kernel; CPU inputs take the
torch reference implementation (triton is CUDA-only, and CPU-side callers
such as unit tests exercise the reference path)."""
for value in (*args, *kwargs.values()):
if isinstance(value, torch.Tensor):
return value.is_cuda
raise AssertionError("kernel dispatch requires at least one tensor argument")
class ExpandPrefillCausallyResult(msgspec.Struct):
seq_lens_casual: torch.Tensor
req_pool_indices_repeated: torch.Tensor
class ExpandPrefillCausally:
@classmethod
def execute(cls, *args, **kwargs) -> ExpandPrefillCausallyResult:
if _inputs_on_cuda(*args, **kwargs):
return cls.triton(*args, **kwargs)
return cls.torch(*args, **kwargs)
@classmethod
def torch(
cls,
*,View on GitHub (pinned to 0132848349)