{"record":{"id":"b7b44bc77e5365d3","repo":"sgl-project/sglang","slug":"kv-canary-at-most-consts-max-real-kv-sources-re","errorCode":null,"errorMessage":"kv-canary: at most {consts.MAX_REAL_KV_SOURCES} RealKvSource entries supported by the CUDA ABI, got {len(real_kv_sources)}","messagePattern":"kv-canary: at most (.+?) RealKvSource entries supported by the CUDA ABI, got (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/kv_canary/verify.py","lineNumber":309,"sourceCode":"        - Counters: each thread maintains a local count of active entries it processed, warp-reduces via\n          ``__shfl_down_sync`` (offsets 16..1), then the warp leader (lane 0) does a single atomicAdd of the\n          warp's summed count into slot_run_counter. kernel_run_counter += 1: single thread (tid == 0) does an\n          atomicAdd once per launch.\n\n    Calling contract:\n        - Pure side-effect; never raises. Host polls violation_write_index[0] > 0 for is_errored and\n          violation_ring[0] for the first violation.\n        - kernel_run_counter is bumped every call (canary-ran health signal).\n        - Safe in cuda-graph capture; caller refills plan in-place before replay.\n\n    Pinned by torch reference\n    :func:`sglang.kernels.ops.kv_canary.verify_ref.launch_canary_verify_kernel_torch_reference`; CUDA must match\n    byte-for-byte.\n    \"\"\"\n    canary_buf = context.canary_buf\n    real_kv_sources = context.real_kv_sources\n    if len(real_kv_sources) > consts.MAX_REAL_KV_SOURCES:\n        raise ValueError(\n            f\"kv-canary: at most {consts.MAX_REAL_KV_SOURCES} RealKvSource entries supported by the CUDA ABI, \"\n            f\"got {len(real_kv_sources)}\"\n        )\n\n    _assert_contiguous(canary_buf, \"canary_buf\")\n    _assert_contiguous(plan.verify_slot_indices, \"plan.verify_slot_indices\")\n    _assert_contiguous(plan.verify_expected_tokens, \"plan.verify_expected_tokens\")\n    _assert_contiguous(plan.verify_expected_positions, \"plan.verify_expected_positions\")\n    _assert_contiguous(plan.verify_prev_slot_indices, \"plan.verify_prev_slot_indices\")\n    _assert_contiguous(plan.verify_num_valid, \"plan.verify_num_valid\")\n    _assert_contiguous(plan.enable, \"plan.enable\")\n    _assert_contiguous(context.violation_ring, \"violation_ring\")\n    _assert_contiguous(context.violation_write_index, \"violation_write_index\")\n    _assert_contiguous(context.slot_run_counter, \"slot_run_counter\")\n    _assert_contiguous(context.kernel_run_counter, \"kernel_run_counter\")\n\n    padded_bufs, source_params = _build_real_kv_source_abi(\n        real_kv_sources=real_kv_sources, device=canary_buf.device","sourceCodeStart":291,"sourceCodeEnd":327,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/kv_canary/verify.py#L291-L327","documentation":"The CUDA kernel ABI for canary verification fixes a static upper bound (consts.MAX_REAL_KV_SOURCES) on how many RealKvSource entries can be packed into the kernel parameter buffer. Passing more entries would overflow the ABI struct, so the Python launcher rejects the list up front.","triggerScenarios":"Calling launch_canary_verify_kernel with a CanaryVerifyContext whose real_kv_sources list has more than consts.MAX_REAL_KV_SOURCES entries (typically >4).","commonSituations":"Wiring a canary harness to several KV pools / hybrid-cache sources (e.g. hierarchical cache layers + host cache) and exceeding the compiled limit; bumping the number of sources in a test sweep without recompiling the kernel with a larger ABI bound.","solutions":["Reduce real_kv_sources to at most consts.MAX_REAL_KV_SOURCES entries (drop or merge the least important sources)","If more sources are genuinely needed, raise MAX_REAL_KV_SOURCES in consts and rebuild the matching CUDA kernel so the ABI agrees","Write a pre-flight check in your harness: assert len(ctx.real_kv_sources) <= consts.MAX_REAL_KV_SOURCES before launching"],"exampleFix":"# before\nctx = CanaryVerifyContext(real_kv_sources=[s1, s2, s3, s4, s5], ...)\nlaunch_canary_verify_kernel(ctx, plan)\n# after\nassert len(sources) <= consts.MAX_REAL_KV_SOURCES\nctx = CanaryVerifyContext(real_kv_sources=sources[:consts.MAX_REAL_KV_SOURCES], ...)\nlaunch_canary_verify_kernel(ctx, plan)","handlingStrategy":"validation","validationCode":"from sglang.kernels.ops.kv_canary import consts\nassert len(ctx.real_kv_sources) <= consts.MAX_REAL_KV_SOURCES\nlaunch_canary_verify_kernel(ctx, plan)","typeGuard":"def sources_within_abi_limit(sources: list) -> bool:\n    return len(sources) <= consts.MAX_REAL_KV_SOURCES","tryCatchPattern":"try:\n    launch_canary_verify_kernel(ctx, plan)\nexcept ValueError as e:\n    if 'MAX_REAL_KV_SOURCES' in str(e):\n        ctx.real_kv_sources = ctx.real_kv_sources[:consts.MAX_REAL_KV_SOURCES]\n    else:\n        raise","preventionTips":["Bound source lists at construction time","Re-run kernel build after changing consts.MAX_REAL_KV_SOURCES"],"tags":["kv-canary","cuda-abi","limit-exceeded"],"backgroundTag":"static-array-limit-exceeded","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}