{"record":{"id":"7bae6b4aa2805746","repo":"sgl-project/sglang","slug":"st-upper-state-index-length-mismatch-prefill","errorCode":null,"errorMessage":"{st.upper()} state index length mismatch: prefill={len(src_indices)}, dst={len(dst_indices_local)}","messagePattern":"(.+?) state index length mismatch: prefill=(.+?), dst=(.+?)","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"critical","filePath":"python/sglang/srt/disaggregation/mooncake/conn.py","lineNumber":1361,"sourceCode":"                ):\n                    raise RuntimeError(\n                        f\"PD Disaggregation does NOT support PD different TP sizes for non-MLA {st.upper()} hybrid models yet.\"\n                    )\n                src_indices = list(indices)\n                dst_indices_local = list(dst_indices)\n                if (\n                    st == StateType.C128_STATE\n                    and len(src_indices) == 0\n                    and len(dst_indices_local) == 0\n                ):\n                    continue\n                if len(src_indices) != len(dst_indices_local):\n                    # These components are position- or request-indexed:\n                    # truncating silently misaligns rows and corrupts KV.\n                    # Paged SWA/DSA tolerate a 1-page drift -> keep the\n                    # lenient truncation below.\n                    if self._requires_exact_state_index_match(st):\n                        raise RuntimeError(\n                            f\"{st.upper()} state index length mismatch: \"\n                            f\"prefill={len(src_indices)}, dst={len(dst_indices_local)}\"\n                        )\n                    logger.warning(\n                        f\"len(prefill_state_indices) = {len(src_indices)}, len(dst_state_indices) = {len(dst_indices_local)}\"\n                    )\n                    if len(src_indices) > len(dst_indices_local):\n                        src_indices = src_indices[: len(dst_indices_local)]\n                    else:\n                        dst_indices_local = dst_indices_local[: len(src_indices)]\n                rc = (\n                    self._send_kvcache_generic(\n                        mooncake_session_id=req.mooncake_session_id,\n                        src_data_ptrs=src_data_ptrs,\n                        dst_data_ptrs=dst_data_ptrs,\n                        item_lens=src_item_lens,\n                        prefill_data_indices=np.array(src_indices, dtype=np.int32),\n                        dst_data_indices=np.array(dst_indices_local, dtype=np.int32),","sourceCodeStart":1343,"sourceCodeEnd":1379,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/disaggregation/mooncake/conn.py#L1343-L1379","documentation":"Extra-state index lists (e.g. position/request-indexed components like C128_STATE) must match in length between prefill and decode; truncation would silently misalign rows and corrupt KV, so _requires_exact_state_index_match types raise instead of warn.","triggerScenarios":"len(src_indices) != len(dst_indices_local) for an exactly-matched StateType during maybe_send_extra.","commonSituations":"Allocator drift producing different row counts per rank, scheduler divergence between PD sides, or earlier silent truncation bugs now surfaced.","solutions":["Check scheduler/allocator logs for why state index counts diverged (failed allocs, preemption on one side)","Restart both instances to reset allocator state","Ensure identical chunking/request scheduling config on both sides"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"catch RuntimeError mentioning 'index length mismatch'; drop the room's requests and alert — do not retry the same corrupted state","preventionTips":["Watch allocator preemption/failure counters on both sides","Alert on any state-index divergence before corruption spreads"],"tags":["pd-disagg","state-index","kv-corruption-guard"],"backgroundTag":"index-length-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}