sgl-project/sglang · error · NotImplementedError

move_kv_cache is not yet supported for MiniMaxSparseKVPool:

Error message

move_kv_cache is not yet supported for MiniMaxSparseKVPool: sub-pools must be built with enable_kv_cache_copy=True first.

What it means

MiniMaxSparseKVPool (a wrapper around per-layer sub-pools for the MiniMax sparse attention model) does not implement move_kv_cache. The method unconditionally raises NotImplementedError because moving KV cache locations requires the sub-pools to be constructed with enable_kv_cache_copy=True so their tensors can be reindexed/copied safely.

Source

Thrown at python/sglang/srt/mem_cache/memory_pool.py:5164

        return self.main_pool.get_contiguous_buf_infos()

    def get_index_k_state_buf_infos(self):
        # Per-page item_len (MHATokenToKVPool convention); index rows share the
        # main-KV `loc`, so the transfer reuses the same page-ids.
        pool = self.index_k_pool
        n = pool.layer_num
        data_ptrs = [pool.k_buffer[i].data_ptr() for i in range(n)]
        data_lens = [pool.k_buffer[i].nbytes for i in range(n)]
        item_lens = [pool.k_buffer[i][0].nbytes * pool.page_size for i in range(n)]
        return data_ptrs, data_lens, item_lens

    def maybe_get_custom_mem_pool(self):
        return self.main_pool.maybe_get_custom_mem_pool()

    def move_kv_cache(self, tgt_loc: torch.Tensor, src_loc: torch.Tensor):
        # TODO: spec-decode needs sub-pools built with enable_kv_cache_copy=True,
        # then delegate to main_pool/index_pool.move_kv_cache.
        raise NotImplementedError(
            "move_kv_cache is not yet supported for MiniMaxSparseKVPool: "
            "sub-pools must be built with enable_kv_cache_copy=True first."
        )

    def get_v_head_dim(self):
        # Use start_layer to handle pipeline parallelism where layer 0
        # may not be present in this stage's buffer.
        return self.main_pool.get_value_buffer(self.main_pool.start_layer).shape[-1]

View on GitHub (pinned to 0132848349)

Solutions

  1. Disable speculative decoding (remove --speculative-algorithm) when serving MiniMaxSparseKVPool models
  2. Avoid/disable features that call move_kv_cache (e.g. certain HiCache/offload eviction strategies) for this model
  3. If you control pool construction, build the sub-pools with enable_kv_cache_copy=True and implement delegation to main_pool/index_pool.move_kv_cache as the TODO describes, then upstream the patch
  4. Follow the SGLang repo/issue tracker for MiniMax sparse + spec-decode support status before upgrading

Example fix

# before
pool.move_kv_cache(tgt_loc, src_loc)  # NotImplementedError

# after (caller guard)
if isinstance(pool, MiniMaxSparseKVPool):
    raise RuntimeError("spec-decode not supported for MiniMaxSparseKVPool; disable it")
pool.move_kv_cache(tgt_loc, src_loc)
Defensive patterns

Strategy: type-guard

Validate before calling

from sglang.srt.mem_cache.memory_pool import MiniMaxSparseKVPool
if isinstance(pool, MiniMaxSparseKVPool):
    raise RuntimeError("move_kv_cache unsupported for MiniMaxSparseKVPool")

Type guard

def supports_move_kv_cache(pool) -> bool:
    return not isinstance(pool, MiniMaxSparseKVPool)

Try / catch

try:
    pool.move_kv_cache(tgt, src)
except NotImplementedError as e:
    logger.warning("KV move unsupported (%s); disabling spec-decode path", e)
    self.spec_algorithm = None

Prevention

When it happens

Trigger: Calling move_kv_cache(tgt_loc, src_loc) on a MiniMaxSparseKVPool instance — typically triggered by SGLang's speculative decoding (e.g. EAGLE-style draft/verify) or hierarchical cache offload/eviction paths that relocate KV entries after a batch runs.

Common situations: Running the MiniMax-M1/MiniMax sparse model together with --speculative-algorithm or hierarchical cache features; hitting the TODO in the source before sub-pool support for KV copy was added.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/1b62834ea4ac050c. Report an issue: GitHub.