{"record":{"id":"bc74cd38ae04002d","repo":"sgl-project/sglang","slug":"dsv4-ragged-verify-does-not-support-online-c128-mt","errorCode":null,"errorMessage":"DSV4 ragged verify does not support online c128 MTP; set SGLANG_RAGGED_VERIFY_MODE off or disable online compress.","messagePattern":"DSV4 ragged verify does not support online c128 MTP; set SGLANG_RAGGED_VERIFY_MODE off or disable online compress\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"critical","filePath":"python/sglang/srt/layers/attention/deepseek_v4_backend.py","lineNumber":647,"sourceCode":"        return pin_tensor.to(self.device, non_blocking=True)\n\n    def _resolve_verify_layout(\n        self,\n        forward_batch: ForwardBatch,\n        bs: int,\n    ) -> Optional[RaggedVerifyLayout]:\n        layout = resolve_ragged_verify_layout(forward_batch)\n        if layout is None:\n            return None\n        if read_ragged_verify_mode() is not RaggedVerifyMode.COMPACT:\n            return None\n        if get_parallel().attn_cp_size > 1:\n            raise NotImplementedError(\n                \"DSV4 ragged verify does not support context parallel (CP); \"\n                \"set SGLANG_RAGGED_VERIFY_MODE off for CP runs.\"\n            )\n        if self.online_c128_mtp.enabled():\n            raise NotImplementedError(\n                \"DSV4 ragged verify does not support online c128 MTP; \"\n                \"set SGLANG_RAGGED_VERIFY_MODE off or disable online compress.\"\n            )\n        # Layout invariants (verify_lens >= 1, total == sum) are enforced in\n        # RaggedVerifyLayout.__post_init__; don't re-check the device tensor\n        # here -- that would D2H-sync the host-free verify prep path.\n        layout = layout.padded_to_bucket(padded_bs=bs)\n        return layout\n\n    def _target_verify_graph_key(\n        self,\n        bs: int,\n        ragged_layout: Optional[RaggedVerifyLayout],\n    ) -> Tuple[int, int]:\n        return compute_target_verify_graph_key(\n            bs=bs,\n            num_draft_tokens=self.speculative_num_draft_tokens,\n            ragged_layout=ragged_layout,","sourceCodeStart":629,"sourceCodeEnd":665,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/attention/deepseek_v4_backend.py#L629-L665","documentation":"The DSV4 ragged-verify layout is incompatible with the online c128 MTP path (self.online_c128_mtp.enabled()). When both are active the backend raises during verify metadata resolution because the compact ragged layout cannot express the online-compressed MTP tokens.","triggerScenarios":"Ragged verify mode COMPACT active (via SGLANG_RAGGED_VERIFY_MODE) and a forward batch for which a ragged verify layout resolves, while online c128 MTP / online compress is enabled on the model runner.","commonSituations":"Enabling ragged verify env var on a DeepSeek-V4 deployment that already uses online compression for MTP draft tokens.","solutions":["Disable SGLANG_RAGGED_VERIFY_MODE (set off)","Disable online compress / online c128 MTP if ragged verify is needed"],"exampleFix":"# before\nSGLANG_RAGGED_VERIFY_MODE=compact  # with online c128 MTP enabled\n# after\nunset SGLANG_RAGGED_VERIFY_MODE","handlingStrategy":"validation","validationCode":"import os\nif online_c128_mtp.enabled() and os.environ.get(\"SGLANG_RAGGED_VERIFY_MODE\", \"\").lower() == \"compact\":\n    del os.environ[\"SGLANG_RAGGED_VERIFY_MODE\"]","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Document which env vars are compatible with online compress/MTP features","Clear experimental env vars from base images before adding new features"],"tags":["deepseek-v4","ragged-verify","mtp","online-compress","env-var","sglang"],"backgroundTag":"unsupported-feature-combination","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}