{"record":{"id":"5677cf6cee9b13d4","repo":"sgl-project/sglang","slug":"store-kv-token-ids-has-n-entries-but-kv-indices","errorCode":null,"errorMessage":"store_kv: token_ids has {n} entries but kv_indices has {len(kv_indices)} entries","messagePattern":"store_kv: token_ids has (.+?) entries but kv_indices has (.+?) entries","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/mem_cache/storage/flexkv/flexkv_connector.py","lineNumber":472,"sourceCode":"        rid: str,\n        token_ids: List[int],\n        kv_indices: torch.Tensor,\n    ) -> int:\n        \"\"\"Schedule a write back from GPU into FlexKV.\n\n        On the sync leader this runs ``put_match`` to discover which\n        tokens are NOT yet in FlexKV's CPU cache (= the \"unmatched\"\n        slice), then ``launch`` on those. On non-leaders the unmatched\n        mask is received over the PP fan-out so cross-node PP can\n        forward its slot mappings.\n\n        Returns the FlexKV task id of the in-flight store, or -1 if\n        nothing needed to be written.\n        \"\"\"\n        token_ids_np = np.asarray(token_ids, dtype=np.int64)\n        n = len(token_ids_np)\n        if n != len(kv_indices):\n            raise ValueError(\n                f\"store_kv: token_ids has {n} entries but kv_indices \"\n                f\"has {len(kv_indices)} entries\"\n            )\n\n        # Page-align inputs *before* put_match so the FlexKV allocator\n        # only reserves slots that line up with the slot_mapping we send.\n        if self.page_size > 1:\n            aligned_len = (n // self.page_size) * self.page_size\n            if aligned_len == 0:\n                self._send_pp_put_meta(-1, [])\n                return -1\n            if aligned_len < n:\n                token_ids_np = token_ids_np[:aligned_len]\n                kv_indices = kv_indices[:aligned_len]\n\n        fkv_task_id = -1\n        if self._sync_ctx.is_sync_leader and self.kv_manager is not None:\n            try:","sourceCodeStart":454,"sourceCodeEnd":490,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/mem_cache/storage/flexkv/flexkv_connector.py#L454-L490","documentation":"store_kv requires token_ids and kv_indices to be parallel arrays: one kv slot index per token. Before doing any page-alignment or put_match work it validates lengths and raises ValueError on mismatch, since a mismatched pair would write KV data under wrong slot mappings.","triggerScenarios":"Calling store_kv(req, token_ids, kv_indices) where the radix cache produced fewer/more indices than tokens — e.g. prefix-cached tokens excluded from indices but included in token_ids, duplicated slots for page alignment done by the caller, or off-by-one slicing of either list.","commonSituations":"Custom offloading code building kv_indices from req.out_cache_loc vs req.origin_input_ids mismatch; page-aligned index expansion done before the connector's own alignment; changes to radix cache trim semantics between versions.","solutions":["Log both lists where store_kv is called and find which caller produced the skew (usually out_cache_loc vs origin_input_ids/filter_indices)","Build kv_indices with the same filtering applied to token_ids (both derive from the same decoded token sequence)","Re-run the failing request with --disable-radix-cache to confirm cache filtering is the cause, then fix index derivation"],"exampleFix":"# before\nconnector.store_kv(req, req.origin_input_ids, req.out_cache_loc)\n\n# after\n# keep ids and indices parallel after filtering cached prefix\nids, idx = req.last_node.get_token_ids_and_last_offset()  # example\nconnector.store_kv(req, ids, idx)","handlingStrategy":"validation","validationCode":"assert len(token_ids) == len(kv_indices), (\n    f'parallel arrays required: {len(token_ids)} ids vs {len(kv_indices)} indices'\n)\nconnector.store_kv(req, token_ids, kv_indices)","typeGuard":"def parallel_store_args(token_ids: Sequence[int], kv_indices: Sequence[int]) -> bool:\n    return len(tuple(token_ids)) == len(tuple(kv_indices))","tryCatchPattern":"try:\n    fkv_id = connector.store_kv(req, token_ids, kv_indices)\nexcept ValueError as e:\n    if 'store_kv: token_ids' in str(e):\n        logger.error('skipping store for rid=%s: %s', req.rid, e)  # drop this store, keep serving\n    else:\n        raise","preventionTips":["Derive token_ids and kv_indices from the same filtered token sequence in one place","Add a unit assertion at the call site before releasing changes to offload logic"],"tags":["flexkv","kv-cache","validation","off-by-one"],"backgroundTag":"input-length-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}