sgl-project/sglang · error · NotImplementedError

NIXL PD transfer does not support HiSparse combined with dec

Error message

NIXL PD transfer does not support HiSparse combined with decode-only speculative decoding.

What it means

Raised in _prepare_payload_xfer when the decode (destination) side registers a heterogeneous mix of KV memory kinds (VRAM + DRAM, i.e. HiSparse/hierarchical cache) AND the decode instance runs speculative decoding while prefill does not (n_dst > n_src, 'decode-only spec dec'). NIXL PD transfer cannot build descriptor lists for mixed-memory destinations with the extra speculative KV regions, so it refuses with NotImplementedError.

Source

Thrown at python/sglang/srt/disaggregation/nixl/conn.py:1009

            peer_info.dst_homogeneous_mem_kind = dst_mem_kind
            peer_info.dcp_token_item_lens = self.prepare_dcp_token_item_lens(
                dst_kv_item_lens
            )
            return

        if (
            self.is_mla_backend
            or self.is_hybrid_mla_backend
            or peer_info.decode_tp_size == self.attn_tp_size
        ):
            dst_mem_kind = None
            try:
                dst_mem_kind = _homogeneous_kv_mem_kind(
                    peer_info.dst_kv_mem_kinds, "destination"
                )
            except NotImplementedError:
                if decode_only_spec_dec:
                    raise NotImplementedError(
                        "NIXL PD transfer does not support HiSparse combined with "
                        "decode-only speculative decoding."
                    )
                mem_segments = _kv_xfer_mem_segments(
                    self.kv_args.kv_data_mem_kinds, peer_info.dst_kv_mem_kinds
                )
                if not mem_segments:
                    raise ValueError("NIXL KV transfer has no KV memory segments")
                self._init_mixed_equal_tp_prep_handles(peer_info, mem_segments)
                return

            if decode_only_spec_dec and dst_mem_kind != "VRAM":
                raise NotImplementedError(
                    "NIXL PD transfer does not support HiSparse combined with "
                    "decode-only speculative decoding."
                )

            peer_info.dst_homogeneous_mem_kind = dst_mem_kind

View on GitHub (pinned to 0132848349)

Solutions

1. Disable speculative decoding on the decode instance (align spec-dec config between prefill and decode)
2. Disable the hierarchical/HiSparse KV cache on the decode instance so all destination KV tensors are VRAM (homogeneous)
3. Run prefill with the same speculative decoding setup so n_dst == n_src (decode-only spec dec no longer triggers)
4. Upgrade SGLang once mixed-memory NIXL transfer with spec-dec support lands

Example fix

# before (decode server)
--disaggregation decode --speculative-algorithm EAGLE --enable-hierarchical-cache
# after
--disaggregation decode --enable-hierarchical-cache   # no spec dec, or drop hi-cache

def preflight_check(src_args, dst_args):
    mixed = len(set(dst_args.kv_data_mem_kinds)) > 1
    spec_dec_mismatch = (src_args.speculative_algorithm is None
                         and dst_args.speculative_algorithm is not None)
    return not (mixed and spec_dec_mismatch)

try:
    conn.add_transfer_request(...)
except NotImplementedError as e:
    if 'HiSparse' in str(e):
        logger.error('Disable spec-dec on decode or hicache; retrying without spec dec')
    raise

When it happens

Trigger: Prefill node without speculative decoding + decode node with speculative decoding enabled, where decode's kv_data_mem_kinds mixes VRAM and DRAM (HiSparse/hierarchical KV cache enabled on decode), so _homogeneous_kv_mem_kind raises NotImplementedError for the destination and decode_only_spec_dec is True.

Common situations: Enabling --speculative-algorithm (e.g. EAGLE) only on the decode server of a PD-disaggregated cluster while also enabling hierarchical cache (HiCache/HiSparse with DRAM tier) on the decode side.

Related errors

• NIXL KV transfer has no KV memory segments
• NIXL heterogeneous-TP direct-to-host KV transfer is not impl
• --disaggregation-decode-enable-radix-cache is incompatible w
• Ascend PD transfer does not support HiSparse destination dev
• Unsupported KV cache type for decode offload

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/b774bfdff5235dcd. Report an issue: GitHub.