sgl-project/sglang · error · ValueError

store_kv: token_ids has {n} entries but kv_indices has {len(

Error message

store_kv: token_ids has {n} entries but kv_indices has {len(kv_indices)} entries

What it means

store_kv requires token_ids and kv_indices to be parallel arrays: one kv slot index per token. Before doing any page-alignment or put_match work it validates lengths and raises ValueError on mismatch, since a mismatched pair would write KV data under wrong slot mappings.

Source

Thrown at python/sglang/srt/mem_cache/storage/flexkv/flexkv_connector.py:472

        rid: str,
        token_ids: List[int],
        kv_indices: torch.Tensor,
    ) -> int:
        """Schedule a write back from GPU into FlexKV.

        On the sync leader this runs ``put_match`` to discover which
        tokens are NOT yet in FlexKV's CPU cache (= the "unmatched"
        slice), then ``launch`` on those. On non-leaders the unmatched
        mask is received over the PP fan-out so cross-node PP can
        forward its slot mappings.

        Returns the FlexKV task id of the in-flight store, or -1 if
        nothing needed to be written.
        """
        token_ids_np = np.asarray(token_ids, dtype=np.int64)
        n = len(token_ids_np)
        if n != len(kv_indices):
            raise ValueError(
                f"store_kv: token_ids has {n} entries but kv_indices "
                f"has {len(kv_indices)} entries"
            )

        # Page-align inputs *before* put_match so the FlexKV allocator
        # only reserves slots that line up with the slot_mapping we send.
        if self.page_size > 1:
            aligned_len = (n // self.page_size) * self.page_size
            if aligned_len == 0:
                self._send_pp_put_meta(-1, [])
                return -1
            if aligned_len < n:
                token_ids_np = token_ids_np[:aligned_len]
                kv_indices = kv_indices[:aligned_len]

        fkv_task_id = -1
        if self._sync_ctx.is_sync_leader and self.kv_manager is not None:
            try:

View on GitHub (pinned to 0132848349)

Solutions

Log both lists where store_kv is called and find which caller produced the skew (usually out_cache_loc vs origin_input_ids/filter_indices)
Build kv_indices with the same filtering applied to token_ids (both derive from the same decoded token sequence)
Re-run the failing request with --disable-radix-cache to confirm cache filtering is the cause, then fix index derivation

Example fix

# before
connector.store_kv(req, req.origin_input_ids, req.out_cache_loc)

# after
# keep ids and indices parallel after filtering cached prefix
ids, idx = req.last_node.get_token_ids_and_last_offset()  # example
connector.store_kv(req, ids, idx)

Defensive patterns

Strategy: validation

Validate before calling

assert len(token_ids) == len(kv_indices), (
    f'parallel arrays required: {len(token_ids)} ids vs {len(kv_indices)} indices'
)
connector.store_kv(req, token_ids, kv_indices)

Type guard

def parallel_store_args(token_ids: Sequence[int], kv_indices: Sequence[int]) -> bool:
    return len(tuple(token_ids)) == len(tuple(kv_indices))

Try / catch

try:
    fkv_id = connector.store_kv(req, token_ids, kv_indices)
except ValueError as e:
    if 'store_kv: token_ids' in str(e):
        logger.error('skipping store for rid=%s: %s', req.rid, e)  # drop this store, keep serving
    else:
        raise

Prevention

Derive token_ids and kv_indices from the same filtered token sequence in one place
Add a unit assertion at the call site before releasing changes to offload logic

When it happens

Trigger: Calling store_kv(req, token_ids, kv_indices) where the radix cache produced fewer/more indices than tokens — e.g. prefix-cached tokens excluded from indices but included in token_ids, duplicated slots for page alignment done by the caller, or off-by-one slicing of either list.

Common situations: Custom offloading code building kv_indices from req.out_cache_loc vs req.origin_input_ids mismatch; page-aligned index expansion done before the connector's own alignment; changes to radix cache trim semantics between versions.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/5677cf6cee9b13d4. Report an issue: GitHub.