sgl-project/sglang · critical · NotImplementedError

DSV4 ragged verify does not support online c128 MTP; set SGL

Error message

DSV4 ragged verify does not support online c128 MTP; set SGLANG_RAGGED_VERIFY_MODE off or disable online compress.

What it means

The DSV4 ragged-verify layout is incompatible with the online c128 MTP path (self.online_c128_mtp.enabled()). When both are active the backend raises during verify metadata resolution because the compact ragged layout cannot express the online-compressed MTP tokens.

Source

Thrown at python/sglang/srt/layers/attention/deepseek_v4_backend.py:647

        return pin_tensor.to(self.device, non_blocking=True)

    def _resolve_verify_layout(
        self,
        forward_batch: ForwardBatch,
        bs: int,
    ) -> Optional[RaggedVerifyLayout]:
        layout = resolve_ragged_verify_layout(forward_batch)
        if layout is None:
            return None
        if read_ragged_verify_mode() is not RaggedVerifyMode.COMPACT:
            return None
        if get_parallel().attn_cp_size > 1:
            raise NotImplementedError(
                "DSV4 ragged verify does not support context parallel (CP); "
                "set SGLANG_RAGGED_VERIFY_MODE off for CP runs."
            )
        if self.online_c128_mtp.enabled():
            raise NotImplementedError(
                "DSV4 ragged verify does not support online c128 MTP; "
                "set SGLANG_RAGGED_VERIFY_MODE off or disable online compress."
            )
        # Layout invariants (verify_lens >= 1, total == sum) are enforced in
        # RaggedVerifyLayout.__post_init__; don't re-check the device tensor
        # here -- that would D2H-sync the host-free verify prep path.
        layout = layout.padded_to_bucket(padded_bs=bs)
        return layout

    def _target_verify_graph_key(
        self,
        bs: int,
        ragged_layout: Optional[RaggedVerifyLayout],
    ) -> Tuple[int, int]:
        return compute_target_verify_graph_key(
            bs=bs,
            num_draft_tokens=self.speculative_num_draft_tokens,
            ragged_layout=ragged_layout,

View on GitHub (pinned to 0132848349)

Solutions

1. Disable SGLANG_RAGGED_VERIFY_MODE (set off)
2. Disable online compress / online c128 MTP if ragged verify is needed

Example fix

# before
SGLANG_RAGGED_VERIFY_MODE=compact  # with online c128 MTP enabled
# after
unset SGLANG_RAGGED_VERIFY_MODE

import os
if online_c128_mtp.enabled() and os.environ.get("SGLANG_RAGGED_VERIFY_MODE", "").lower() == "compact":
    del os.environ["SGLANG_RAGGED_VERIFY_MODE"]

When it happens

Trigger: Ragged verify mode COMPACT active (via SGLANG_RAGGED_VERIFY_MODE) and a forward batch for which a ragged verify layout resolves, while online c128 MTP / online compress is enabled on the model runner.

Common situations: Enabling ragged verify env var on a DeepSeek-V4 deployment that already uses online compression for MTP draft tokens.

Related errors

• DSV4 ragged verify does not support context parallel (CP); s
• Invalid ltx2_two_stage_device_mode={mode!r}. Expected one of
• --enable-hisparse is not supported with the unified-KV path
• Kimi-K3 DCP + DSPARK currently requires SGLANG_RAGGED_VERIFY
• DSpark with dp attention + moe_a2a_backend={} requires SGLAN

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/bc74cd38ae04002d. Report an issue: GitHub.