{"record":{"id":"41fc9eeb7872f72c","repo":"sgl-project/sglang","slug":"head-slice-size-evaluates-to-zero","errorCode":null,"errorMessage":"Head slice size evaluates to zero","messagePattern":"Head slice size evaluates to zero","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"critical","filePath":"python/sglang/srt/disaggregation/mori/conn.py","lineNumber":932,"sourceCode":"        src_item_len = self.kv_args.kv_item_lens[0]\n        dst_item_len = peer_info.dst_kv_item_len\n\n        bytes_per_token_src = src_item_len // page_size\n        bytes_per_token_dst = dst_item_len // page_size\n\n        prefill_tp_size = self.attn_tp_size\n        decode_tp_size = peer_info.decode_tp_size\n\n        total_kv_heads = getattr(self.kv_args, \"total_kv_head_num\", 0)\n        if total_kv_heads <= 0:\n            total_kv_heads = self.kv_args.kv_head_num * prefill_tp_size\n\n        src_heads_per_rank = max(1, total_kv_heads // prefill_tp_size)\n        dst_heads_per_rank = max(1, total_kv_heads // decode_tp_size)\n\n        bytes_per_head_slice = bytes_per_token_dst // dst_heads_per_rank\n        if bytes_per_head_slice == 0:\n            raise ValueError(\"Head slice size evaluates to zero\")\n\n        src_replication = max(1, prefill_tp_size // total_kv_heads)\n\n        local_tp_rank = self.kv_args.engine_rank % prefill_tp_size\n        dst_tp_rank = peer_info.decode_tp_rank % decode_tp_size\n\n        if prefill_tp_size > decode_tp_size:\n            src_head_start = 0\n            num_heads_to_send = src_heads_per_rank\n            unique_head_idx = local_tp_rank // src_replication\n            dst_head_start = (unique_head_idx * src_heads_per_rank) % dst_heads_per_rank\n        else:\n            src_head_start = (dst_tp_rank * dst_heads_per_rank) % src_heads_per_rank\n            num_heads_to_send = dst_heads_per_rank\n            dst_head_start = 0\n\n        src_head_slice_offset = src_head_start * bytes_per_head_slice\n        dst_head_slice_offset = dst_head_start * bytes_per_head_slice","sourceCodeStart":914,"sourceCodeEnd":950,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/disaggregation/mori/conn.py#L914-L950","documentation":"While building the TP (tensor-parallel) slice configuration for KV transfer, the per-head byte slice computed as bytes_per_token_dst // dst_heads_per_rank evaluates to zero. This means the destination token's byte budget divided among its KV heads yields nothing, typically because total_kv_heads is larger than the per-token byte size or bytes_per_token_dst is zero/miscomputed.","triggerScenarios":"send_kvcache on a TP-mismatched PD setup where bytes_per_token_dst (from the decode-side memory descriptor item length) is smaller than dst_heads_per_rank (total_kv_heads // decode_tp_size), or total_kv_heads exceeds the token byte size, making the integer division zero.","commonSituations":"Head-dimension or dtype mismatch between prefill and decode memory pool registration, corrupted/zero item_len in the exchanged memory descriptors, or an exotic TP/heads combination where heads outnumber bytes per token (very small head_dim with tiny dtype).","solutions":["Verify total_kv_heads matches the model's num_kv_heads and that decode_tp_size divides it reasonably","Check that the decode-side descriptor bytes_per_token_dst (item_len * page_size arithmetic) is nonzero and matches dtype_size * head_dim * heads","Ensure prefill and decode use the same dtype and head_dim for the KV cache","Log bytes_per_token_dst and dst_heads_per_rank before transfer to confirm the arithmetic inputs"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"dst_heads = max(1, total_kv_heads // decode_tp_size)\nassert bytes_per_token_dst > 0 and bytes_per_token_dst // dst_heads > 0, (\n    \"bytes_per_token_dst too small for head slicing\"\n)","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Pin identical KV dtype/head_dim on both PD instances","Sanity-check memory descriptor item_lens after registration","Avoid TP/heads combos where heads exceed per-token bytes"],"tags":["disaggregation","tensor-parallel","kv-cache","integer-division"],"backgroundTag":"invalid-derived-config-value","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}