{"record":{"id":"2d42b1c349e37ee4","repo":"sgl-project/sglang","slug":"kda-num-heads-num-heads-must-be-divisible-by-s","errorCode":null,"errorMessage":"KDA num_heads ({num_heads}) must be divisible by shard tp_size ({tp_size})","messagePattern":"KDA num_heads \\((.+?)\\) must be divisible by shard tp_size \\((.+?)\\)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/models/kimi_linear.py","lineNumber":58,"sourceCode":")\nfrom sglang.srt.model_executor.forward_batch_info import ForwardBatch, PPProxyTensors\nfrom sglang.srt.model_executor.runner import get_is_capture_mode\nfrom sglang.srt.model_loader.weight_utils import (\n    default_weight_loader,\n    maybe_remap_kv_scale_name,\n    sharded_weight_loader,\n)\nfrom sglang.srt.models.deepseek_v2 import DeepseekV2AttentionMLA as KimiMLAAttention\nfrom sglang.srt.models.llama import LlamaMLP as KimiMLP\nfrom sglang.srt.models.transformers import maybe_prefix\nfrom sglang.srt.runtime_context import get_parallel, get_stream\nfrom sglang.srt.utils import is_xpu, make_layers\nfrom sglang.srt.utils.common import BumpAllocator, add_prefix, set_weight_attrs\n\n\ndef _get_kda_local_num_heads(num_heads: int, tp_size: int) -> int:\n    if num_heads % tp_size != 0:\n        raise ValueError(\n            f\"KDA num_heads ({num_heads}) must be divisible by shard tp_size ({tp_size})\"\n        )\n    return num_heads // tp_size\n\n\ndef _materialize_residual_stream(\n    hidden_states: torch.Tensor, residual: Optional[torch.Tensor]\n) -> torch.Tensor:\n    return hidden_states if residual is None else hidden_states + residual\n\n\nclass KimiMoE(nn.Module):\n    def __init__(\n        self,\n        config: KimiLinearConfig,\n        quant_config: Optional[QuantizationConfig] = None,\n        prefix: str = \"\",\n        layer_idx: int = 0,","sourceCodeStart":40,"sourceCodeEnd":76,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/models/kimi_linear.py#L40-L76","documentation":"Kimi-Linear's KDA (Kimi Delta Attention) state shards its num_heads across tensor-parallel ranks; _get_kda_local_num_heads (kimi_linear.py:58) requires num_heads % tp_size == 0 and otherwise raises at model init. Head counts cannot be split unevenly, so TP degrees like 3, 5, 6, 7 are invalid for a given head count.","triggerScenarios":"Launching with --tp-size N where N does not divide the config's KDA num_heads (e.g. tp=6 with 32 heads: 32 % 6 != 0).","commonSituations":"Choosing TP size based on GPU count without checking head divisibility; mixing KDA (linear attention) models with TP values valid only for standard transformers.","solutions":["Pick a tp_size that divides the KDA num_heads from config (commonly powers of 2: 1/2/4/8)","Check config (e.g. num_kda_heads / linear attention heads) before choosing --tp-size","Use expert/data parallelism instead of an odd TP degree if GPU count forces it"],"exampleFix":"# before\npython -m sglang.launch_server --tp-size 6 ...\n# after\npython -m sglang.launch_server --tp-size 4 ...","handlingStrategy":"validation","validationCode":"assert cfg.num_kda_heads % args.tp_size == 0, f\"{cfg.num_kda_heads} % {args.tp_size}\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Always check head-count divisibility before choosing TP degree","Prefer power-of-two TP sizes for attention-bearing models"],"tags":["kimi-linear","tensor-parallel","divisibility","startup-validation"],"backgroundTag":"tp-head-divisibility","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}