sgl-project/sglang · error · ValueError
kv-canary: at most RealKvSource entries supported by the…
Error message
kv-canary: at most {consts.MAX_REAL_KV_SOURCES} RealKvSource entries supported by the CUDA ABI, got {len(real_kv_sources)} What it means
The CUDA kernel ABI for canary verification fixes a static upper bound (consts.MAX_REAL_KV_SOURCES) on how many RealKvSource entries can be packed into the kernel parameter buffer. Passing more entries would overflow the ABI struct, so the Python launcher rejects the list up front.
Solutions
- Reduce real_kv_sources to at most consts.MAX_REAL_KV_SOURCES entries (drop or merge the least important sources)
- If more sources are genuinely needed, raise MAX_REAL_KV_SOURCES in consts and rebuild the matching CUDA kernel so the ABI agrees
- Write a pre-flight check in your harness: assert len(ctx.real_kv_sources) <= consts.MAX_REAL_KV_SOURCES before launching
Example fix
# before ctx = CanaryVerifyContext(real_kv_sources=[s1, s2, s3, s4, s5], ...) launch_canary_verify_kernel(ctx, plan) # after assert len(sources) <= consts.MAX_REAL_KV_SOURCES ctx = CanaryVerifyContext(real_kv_sources=sources[:consts.MAX_REAL_KV_SOURCES], ...) launch_canary_verify_kernel(ctx, plan)
Defensive patterns
Strategy: validation
Validate before calling
from sglang.kernels.ops.kv_canary import consts assert len(ctx.real_kv_sources) <= consts.MAX_REAL_KV_SOURCES launch_canary_verify_kernel(ctx, plan)
Type guard
def sources_within_abi_limit(sources: list) -> bool:
return len(sources) <= consts.MAX_REAL_KV_SOURCES Try / catch
try:
launch_canary_verify_kernel(ctx, plan)
except ValueError as e:
if 'MAX_REAL_KV_SOURCES' in str(e):
ctx.real_kv_sources = ctx.real_kv_sources[:consts.MAX_REAL_KV_SOURCES]
else:
raise Prevention
- Bound source lists at construction time
- Re-run kernel build after changing consts.MAX_REAL_KV_SOURCES
When it happens
Trigger: Calling launch_canary_verify_kernel with a CanaryVerifyContext whose real_kv_sources list has more than consts.MAX_REAL_KV_SOURCES entries (typically >4).
Common situations: Wiring a canary harness to several KV pools / hybrid-cache sources (e.g. hierarchical cache layers + host cache) and exceeding the compiled limit; bumping the number of sources in a test sweep without recompiling the kernel with a larger ABI bound.
Related errors
- kv-canary: at most RealKvSource entries supported by the…
- kv-canary: bs= exceeds write_req_capacity=
- kv-canary: canary_buf slot stride must hold at least 4…
- kv-canary: canary_buf slot stride must hold at least 4…
- kv-canary: cuda_graph_max_bs must be non-negative, got
AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28).
Data as JSON: /api/errors/b7b44bc77e5365d3.
Report an issue: GitHub.
Appendix: source
Thrown at python/sglang/kernels/ops/kv_canary/verify.py:309
- Counters: each thread maintains a local count of active entries it processed, warp-reduces via
``__shfl_down_sync`` (offsets 16..1), then the warp leader (lane 0) does a single atomicAdd of the
warp's summed count into slot_run_counter. kernel_run_counter += 1: single thread (tid == 0) does an
atomicAdd once per launch.
Calling contract:
- Pure side-effect; never raises. Host polls violation_write_index[0] > 0 for is_errored and
violation_ring[0] for the first violation.
- kernel_run_counter is bumped every call (canary-ran health signal).
- Safe in cuda-graph capture; caller refills plan in-place before replay.
Pinned by torch reference
:func:`sglang.kernels.ops.kv_canary.verify_ref.launch_canary_verify_kernel_torch_reference`; CUDA must match
byte-for-byte.
"""
canary_buf = context.canary_buf
real_kv_sources = context.real_kv_sources
if len(real_kv_sources) > consts.MAX_REAL_KV_SOURCES:
raise ValueError(
f"kv-canary: at most {consts.MAX_REAL_KV_SOURCES} RealKvSource entries supported by the CUDA ABI, "
f"got {len(real_kv_sources)}"
)
_assert_contiguous(canary_buf, "canary_buf")
_assert_contiguous(plan.verify_slot_indices, "plan.verify_slot_indices")
_assert_contiguous(plan.verify_expected_tokens, "plan.verify_expected_tokens")
_assert_contiguous(plan.verify_expected_positions, "plan.verify_expected_positions")
_assert_contiguous(plan.verify_prev_slot_indices, "plan.verify_prev_slot_indices")
_assert_contiguous(plan.verify_num_valid, "plan.verify_num_valid")
_assert_contiguous(plan.enable, "plan.enable")
_assert_contiguous(context.violation_ring, "violation_ring")
_assert_contiguous(context.violation_write_index, "violation_write_index")
_assert_contiguous(context.slot_run_counter, "slot_run_counter")
_assert_contiguous(context.kernel_run_counter, "kernel_run_counter")
padded_bufs, source_params = _build_real_kv_source_abi(
real_kv_sources=real_kv_sources, device=canary_buf.deviceView on GitHub (pinned to 0132848349)