sgl-project/sglang · error · RuntimeError

DSV4 target and draft pools must use the same unified-KV mod

Error message

DSV4 target and draft pools must use the same unified-KV mode

What it means

setup_state_kv_args validates that the DSV4 target pool and draft pool were built with the same _unified_kv flag (unified-KV mode on/off). If they differ, the state buffers to transfer have different layouts, so the shared-state transfer path is rejected.

Source

Thrown at python/sglang/srt/disaggregation/utils.py:1151

                c128_lens,
                c128_item_lens,
            )

    # DSV4 NextN shares the target allocator, so target and draft use the same
    # local SWA indices. Keep draft buffers in a separate positional component
    # to avoid mixing them into the target's heterogeneous state layout, while
    # reusing the existing SWA transport dispatch on both GPU and NPU.
    if isinstance(token_to_kv_pool, DeepSeekV4TokenToKVPool) and isinstance(
        draft_token_to_kv_pool, DeepSeekV4TokenToKVPool
    ):
        if not draft_token_to_kv_pool.compression_ratios or not all(
            ratio == 0 for ratio in draft_token_to_kv_pool.compression_ratios
        ):
            raise RuntimeError(
                "DSV4 draft state transfer expects SWA-only NextN layers"
            )
        if token_to_kv_pool._unified_kv != draft_token_to_kv_pool._unified_kv:
            raise RuntimeError(
                "DSV4 target and draft pools must use the same unified-KV mode"
            )

        if token_to_kv_pool._unified_kv:
            target_geometry = (
                token_to_kv_pool.unified_swa_window,
                token_to_kv_pool.unified_swa_ring_size,
                token_to_kv_pool.unified_swa_pages,
            )
            draft_geometry = (
                draft_token_to_kv_pool.unified_swa_window,
                draft_token_to_kv_pool.unified_swa_ring_size,
                draft_token_to_kv_pool.unified_swa_pages,
            )
            if target_geometry != draft_geometry:
                raise RuntimeError(
                    "DSV4 target and draft pools must share SWA ring geometry: "
                    f"target={target_geometry}, draft={draft_geometry}"

View on GitHub (pinned to 0132848349)

Solutions

1. Set the same unified-KV mode for target and draft pools (align the flag/config that sets _unified_kv, e.g. enable it for both or neither)
2. Regenerate/align the draft (NextN) weights config to match the target's unified-KV setting
3. Disable speculative decoding if unified-KV modes cannot be aligned

Example fix

# before
target pool: _unified_kv=True; draft pool: _unified_kv=False
# after
target pool: _unified_kv=True; draft pool: _unified_kv=True

assert target_pool._unified_kv == draft_pool._unified_kv, "unified-KV mode mismatch"

def unified_kv_aligned(a, b) -> bool:
    return a._unified_kv == b._unified_kv

When it happens

Trigger: DSV4 disaggregation + speculative decoding where server args or pool constructors enable unified KV for the target model but not the draft (or vice versa), e.g. differing --enable-unified-kv-style flags applied only to one pool.

Common situations: Draft model config overriding a unified-KV flag; flag applied per-pool with inconsistent values; changing unified-KV default in a newer version affecting only one of the two pools.

Related errors

• DSV4 draft state transfer expects SWA-only NextN layers
• DSV4 target and draft pools must share SWA ring geometry: ta
• DSV4 target and draft pools must share the SWA index mapping
• DSV4 target and draft pools must share paged SWA geometry: t
• num_instances must be >= 1, got {num_instances}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/5e1d79d4eaf0b04e. Report an issue: GitHub.