{"record":{"id":"9b0d4dd273bb44e5","repo":"sgl-project/sglang","slug":"cannot-resolve-total-kv-heads-kv-args-has-neither","errorCode":null,"errorMessage":"Cannot resolve total_kv_heads: kv_args has neither total_kv_head_num nor kv_head_num. Ensure DecodePreallocQueue._init_kv_manager sets kv_args.kv_head_num.","messagePattern":"Cannot resolve total_kv_heads: kv_args has neither total_kv_head_num nor kv_head_num\\. Ensure DecodePreallocQueue\\._init_kv_manager sets kv_args\\.kv_head_num\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/disaggregation/common/staging_buffer.py","lineNumber":766,"sourceCode":"            dst_tp_rank,\n            total_kv_heads,\n        )\n        writer_bytes.append(num_tokens * nh * bytes_per_head_token * num_layers * 2)\n    return num_writers, writer_bytes, sum(writer_bytes)\n\n\ndef resolve_total_kv_heads(\n    kv_args,\n    attn_tp_size: int,\n) -> int:\n    \"\"\"Resolve the global total KV head count from kv_args metadata.\"\"\"\n    total = getattr(kv_args, \"total_kv_head_num\", 0)\n    if total > 0:\n        return total\n    per_rank = getattr(kv_args, \"kv_head_num\", 0)\n    if per_rank > 0:\n        return per_rank * attn_tp_size\n    raise ValueError(\n        \"Cannot resolve total_kv_heads: kv_args has neither total_kv_head_num \"\n        \"nor kv_head_num. \"\n        \"Ensure DecodePreallocQueue._init_kv_manager sets kv_args.kv_head_num.\"\n    )\n\n\ndef staging_grid_tokens(chunked_prefill_size: Optional[int], page_size: int) -> int:\n    \"\"\"Token width of one staging grid slot; shared by prefetch and the\n    sender's grid alignment.\"\"\"\n    cps = chunked_prefill_size or 8192\n    return max(1, cps // page_size) * page_size\n\n\ndef compute_grid_segments(\n    start_idx: int, end_idx: int, base: int, grid_tokens: int\n) -> List[Tuple[int, int]]:\n    \"\"\"Split [start_idx, end_idx) at grid boundaries base + k * grid_tokens\n    so each segment maps to exactly one staging slot. An empty range yields","sourceCodeStart":748,"sourceCodeEnd":784,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/disaggregation/common/staging_buffer.py#L748-L784","documentation":"resolve_total_kv_heads derives the total number of KV heads from kv_args by first reading total_kv_head_num, then falling back to kv_head_num * attn_tp_size. If both attributes are missing/zero, there is not enough metadata to compute per-layer transfer shapes, so it raises with a pointer to DecodePreallocQueue._init_kv_manager.","triggerScenarios":"Calling staging-buffer helpers (create, handle_staging_req, send_kvcache_staged) with a kv_args object lacking both total_kv_head_num and kv_head_num — typically when _init_kv_manager did not populate kv_head_num for a new backend or code path.","commonSituations":"New transfer backend integration that skips setting kv_args.kv_head_num; code paths that construct KVArgs fresh without the prealloc init; refactors dropping the field assignment.","solutions":["Ensure DecodePreallocQueue._init_kv_manager sets kv_args.kv_head_num (per-rank) or kv_args.total_kv_head_num before staging is used","If you build kv_args manually, populate both fields and pass the correct attn_tp_size","Add a startup assertion after kv_manager init to fail fast with clearer context"],"exampleFix":"# before\nkv_args = KVArgs()  # kv_head_num never set\n# after\nkv_args.kv_head_num = total_kv_heads // attn_tp_size\nkv_args.total_kv_head_num = total_kv_heads","handlingStrategy":"validation","validationCode":"assert getattr(kv_args, 'total_kv_head_num', 0) > 0 or getattr(kv_args, 'kv_head_num', 0) > 0, 'kv_args missing kv head counts'","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Populate kv_head_num/total_kv_head_num immediately when building KVArgs","Add a post-init smoke check in backend integration tests"],"tags":["disaggregation","staging-buffer","kv-heads","missing-metadata"],"backgroundTag":"missing-required-field","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}