sgl-project/sglang · error · RuntimeError

DSV4 draft state transfer expects SWA-only NextN layers

Error message

DSV4 draft state transfer expects SWA-only NextN layers

What it means

For DeepSeek-V4 target+draft pool pairs, draft (NextN speculative) state transfer requires every draft layer to be SWA-only, expressed as compression_ratios all equal to 0. Any nonzero compression ratio in the draft pool means the draft model has compressed layers whose state cannot be transferred via the SWA path, so setup_state_kv_args raises RuntimeError.

Source

Thrown at python/sglang/srt/disaggregation/utils.py:1147

            append_state_component(
                kv_args,
                AscendStateType.DSV4_C128,
                c128_ptrs,
                c128_lens,
                c128_item_lens,
            )

    # DSV4 NextN shares the target allocator, so target and draft use the same
    # local SWA indices. Keep draft buffers in a separate positional component
    # to avoid mixing them into the target's heterogeneous state layout, while
    # reusing the existing SWA transport dispatch on both GPU and NPU.
    if isinstance(token_to_kv_pool, DeepSeekV4TokenToKVPool) and isinstance(
        draft_token_to_kv_pool, DeepSeekV4TokenToKVPool
    ):
        if not draft_token_to_kv_pool.compression_ratios or not all(
            ratio == 0 for ratio in draft_token_to_kv_pool.compression_ratios
        ):
            raise RuntimeError(
                "DSV4 draft state transfer expects SWA-only NextN layers"
            )
        if token_to_kv_pool._unified_kv != draft_token_to_kv_pool._unified_kv:
            raise RuntimeError(
                "DSV4 target and draft pools must use the same unified-KV mode"
            )

        if token_to_kv_pool._unified_kv:
            target_geometry = (
                token_to_kv_pool.unified_swa_window,
                token_to_kv_pool.unified_swa_ring_size,
                token_to_kv_pool.unified_swa_pages,
            )
            draft_geometry = (
                draft_token_to_kv_pool.unified_swa_window,
                draft_token_to_kv_pool.unified_swa_ring_size,
                draft_token_to_kv_pool.unified_swa_pages,
            )

View on GitHub (pinned to 0132848349)

Solutions

1. Use an SWA-only NextN draft model (all compression_ratios == 0)
2. Disable speculative decoding for this deployment
3. Disable PD disaggregation and serve single-node

Example fix

# before
--speculative-algorithm NEXTN --disaggregation-prefill  # draft has compressed layers
# after
--speculative-algorithm NEXTN --disaggregation-prefill  # with SWA-only draft (ratios all 0)

if draft_pool.compression_ratios and any(r != 0 for r in draft_pool.compression_ratios):
    raise SystemExit("Draft pool must be SWA-only (compression_ratios all 0) for DSV4 state transfer")

def dsv4_draft_transfer_ok(draft_pool) -> bool:
    return all(r == 0 for r in draft_pool.compression_ratios)

When it happens

Trigger: Running DSV4 with speculative decoding (NextN draft pool is a DeepSeekV4TokenToKVPool) where draft_token_to_kv_pool.compression_ratios contains a nonzero ratio, under PD disaggregation.

Common situations: Enabling MTP/speculative decoding with a draft config that includes compressed layers; mixing a compressed-layer draft with disaggregation; new draft configs after a model/config update.

Related errors

• DSV4 target and draft pools must share SWA ring geometry: ta
• DSV4 target and draft pools must share the SWA index mapping
• PD state transfer does not support TP-mismatched non-MLA SWA
• NIXL PD transfer does not support HiSparse combined with dec
• DSV4 target and draft pools must use the same unified-KV mod

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/3defc1710d3da8d8. Report an issue: GitHub.