{"record":{"id":"8f6de0282921311a","repo":"sgl-project/sglang","slug":"kv-canary-at-most-consts-max-real-kv-sources-re-8f6de0","errorCode":null,"errorMessage":"kv-canary: at most {consts.MAX_REAL_KV_SOURCES} RealKvSource entries supported by the CUDA ABI, got {len(real_kv_sources)}","messagePattern":"kv-canary: at most (.+?) RealKvSource entries supported by the CUDA ABI, got (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/kv_canary/write.py","lineNumber":189,"sourceCode":"        - record_violation() identical to verify (atomicAdd + atomic-write).\n        - Counters: thread of block 0 does atomicAdd(kernel_run_counter, 1); each block accumulates its\n          entry_count and atomicAdds to slot_run_counter once at exit.\n\n    Calling contract:\n        - Pure side-effect; never raises.\n        - Input-verification mismatch records violations but does NOT abort the chain.\n        - kernel_run_counter is bumped every call.\n        - Safe in cuda-graph capture; caller refills input_ids / positions / out_cache_loc / plan\n          in-place before replay.\n\n    Pinned by torch reference\n    :func:`sglang.kernels.ops.kv_canary.write_ref.launch_canary_write_kernel_torch_reference`; CUDA must match\n    byte-for-byte.\n    \"\"\"\n    canary_buf = context.canary_buf\n    real_kv_sources = context.real_kv_sources\n    if len(real_kv_sources) > consts.MAX_REAL_KV_SOURCES:\n        raise ValueError(\n            f\"kv-canary: at most {consts.MAX_REAL_KV_SOURCES} RealKvSource entries supported by the CUDA ABI, \"\n            f\"got {len(real_kv_sources)}\"\n        )\n\n    _assert_contiguous(canary_buf, \"canary_buf\")\n    _assert_contiguous(plan.write_offsets, \"plan.write_offsets\")\n    _assert_contiguous(plan.write_seed_slot_indices, \"plan.write_seed_slot_indices\")\n    _assert_contiguous(plan.write_num_valid_reqs, \"plan.write_num_valid_reqs\")\n    _assert_contiguous(input_ids, \"input_ids\")\n    _assert_contiguous(positions, \"positions\")\n    _assert_contiguous(out_cache_loc, \"out_cache_loc\")\n    if enable_write_input_assert:\n        if expected_input_tokens is None or expected_input_positions is None:\n            raise ValueError(\n                \"kv-canary: expected input tensors are required when enable_write_input_assert=True\"\n            )\n        _assert_contiguous(expected_input_tokens, \"expected_input_tokens\")\n        _assert_contiguous(expected_input_positions, \"expected_input_positions\")","sourceCodeStart":171,"sourceCodeEnd":207,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/kv_canary/write.py#L171-L207","documentation":"The canary write kernel shares the same fixed CUDA ABI parameter table as the verify kernel, which only has room for consts.MAX_REAL_KV_SOURCES RealKvSource descriptors. The Python launcher refuses a longer list before touching the kernel.","triggerScenarios":"Calling launch_canary_write_kernel with context.real_kv_sources longer than consts.MAX_REAL_KV_SOURCES (commonly >4).","commonSituations":"Attaching the canary write path to many KV sources (multiple cache tiers, DP shards folded into one context) and exceeding the compiled bound; tests enumerating sources in a sweep.","solutions":["Trim/merge real_kv_sources to <= consts.MAX_REAL_KV_SOURCES","If the extra sources are required, raise consts.MAX_REAL_KV_SOURCES and rebuild the CUDA extension so the ABI matches","Pre-assert the bound in your harness setup"],"exampleFix":"# before\nctx = CanaryWriteContext(real_kv_sources=all_sources)  # 5 entries\n# after\nctx = CanaryWriteContext(real_kv_sources=all_sources[:consts.MAX_REAL_KV_SOURCES])","handlingStrategy":"validation","validationCode":"from sglang.kernels.ops.kv_canary import consts\nif len(ctx.real_kv_sources) > consts.MAX_REAL_KV_SOURCES:\n    ctx.real_kv_sources = ctx.real_kv_sources[:consts.MAX_REAL_KV_SOURCES]\nlaunch_canary_write_kernel(ctx, plan)","typeGuard":"def sources_within_abi_limit(sources: list) -> bool:\n    return len(sources) <= consts.MAX_REAL_KV_SOURCES","tryCatchPattern":"try:\n    launch_canary_write_kernel(ctx, plan)\nexcept ValueError as e:\n    if 'MAX_REAL_KV_SOURCES' not in str(e):\n        raise\n    ctx.real_kv_sources = ctx.real_kv_sources[:consts.MAX_REAL_KV_SOURCES]\n    launch_canary_write_kernel(ctx, plan)","preventionTips":["Enforce the ABI bound where the context is built","Keep verify and write contexts sharing one bounded source list"],"tags":["kv-canary","cuda-abi","limit-exceeded"],"backgroundTag":"static-array-limit-exceeded","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}