sgl-project/sglang · error · ValueError

NIXL KV transfer has no KV memory segments

Error message

NIXL KV transfer has no KV memory segments

What it means

Raised when _kv_xfer_mem_segments returns an empty list, meaning no valid source/destination memory-kind pairing exists for the mixed-memory (HiSparse) equal-TP/MLA transfer path. The destination KV memory kinds are heterogeneous but none of the supported src-kind/dst-kind segment combinations match, so there are no transferable KV segments.

Source

Thrown at python/sglang/srt/disaggregation/nixl/conn.py:1017

            or self.is_hybrid_mla_backend
            or peer_info.decode_tp_size == self.attn_tp_size
        ):
            dst_mem_kind = None
            try:
                dst_mem_kind = _homogeneous_kv_mem_kind(
                    peer_info.dst_kv_mem_kinds, "destination"
                )
            except NotImplementedError:
                if decode_only_spec_dec:
                    raise NotImplementedError(
                        "NIXL PD transfer does not support HiSparse combined with "
                        "decode-only speculative decoding."
                    )
                mem_segments = _kv_xfer_mem_segments(
                    self.kv_args.kv_data_mem_kinds, peer_info.dst_kv_mem_kinds
                )
                if not mem_segments:
                    raise ValueError("NIXL KV transfer has no KV memory segments")
                self._init_mixed_equal_tp_prep_handles(peer_info, mem_segments)
                return

            if decode_only_spec_dec and dst_mem_kind != "VRAM":
                raise NotImplementedError(
                    "NIXL PD transfer does not support HiSparse combined with "
                    "decode-only speculative decoding."
                )

            peer_info.dst_homogeneous_mem_kind = dst_mem_kind
            # Build the shared src dlist on the first equal-TP/MLA peer; later
            # peers reuse it. Skipped entirely on heterogeneous-TP-only setups.
            if "" not in self.prep_handles:
                self._init_equal_tp_prep_handle(
                    "",
                    self.kv_args.kv_data_ptrs,
                    self.kv_args.gpu_id,
                    mem_kind=src_mem_kind,

View on GitHub (pinned to 0132848349)

Solutions

1. Make prefill and decode use the same KV cache memory configuration (both VRAM-only or both hierarchical with the same layout)
2. Check kv_data_mem_kinds on both sides at startup and align server args (--enable-hierarchical-cache, hi-cache ratio settings)
3. Update both PD nodes to the same SGLang version so mem-kind negotiation matches
4. If hierarchical cache is not needed, disable it on both sides

Example fix

# before
prefill:  --enable-hierarchical-cache --hicache-ratio 2.0
decode:   (no hicache)
# after
prefill:  --enable-hierarchical-cache --hicache-ratio 2.0
decode:   --enable-hierarchical-cache --hicache-ratio 2.0

from sglang.srt.disaggregation.nixl.conn import _kv_xfer_mem_segments
assert len(_kv_xfer_mem_segments(src.kv_data_mem_kinds, dst.kv_data_mem_kinds)) > 0, 'no transferable KV segments'

except ValueError as e:
    if 'no KV memory segments' in str(e):
        fail_cluster_preflight('misaligned hicache configs between prefill and decode')

When it happens

Trigger: Equal-TP or MLA/hybrid-MLA backend where _homogeneous_kv_mem_kind raises NotImplementedError (mixed dst mem kinds), decode_only_spec_dec is False, and _kv_xfer_mem_segments(self.kv_args.kv_data_mem_kinds, peer_info.dst_kv_mem_kinds) yields no segments — e.g. a source memory kind with no compatible destination segment.

Common situations: Mismatched hierarchical-cache configurations between prefill and decode nodes (one VRAM-only, the other with a DRAM tier); version drift where one side reports mem kinds the other cannot pair.

Related errors

• NIXL PD transfer does not support HiSparse combined with dec
• NIXL heterogeneous-TP direct-to-host KV transfer is not impl
• NIXL memory registration failed for {mem_kind} kv tensors
• NIXL memory registration failed for aux tensors
• NIXL memory registration failed for state tensors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/26463008a9e86cb7. Report an issue: GitHub.