{"record":{"id":"58b82c7979e9032e","repo":"sgl-project/sglang","slug":"dsv4-ragged-verify-does-not-support-context-parall","errorCode":null,"errorMessage":"DSV4 ragged verify does not support context parallel (CP); set SGLANG_RAGGED_VERIFY_MODE off for CP runs.","messagePattern":"DSV4 ragged verify does not support context parallel \\(CP\\); set SGLANG_RAGGED_VERIFY_MODE off for CP runs\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"critical","filePath":"python/sglang/srt/layers/attention/deepseek_v4_backend.py","lineNumber":642,"sourceCode":"        self._verify_mask = None\n        self.cuda_graph_swa_out_cache_loc: Optional[torch.Tensor] = None\n\n    def _move_to_device(self, x: List[int]) -> torch.Tensor:\n        pin_tensor = torch.tensor(x, dtype=torch.int32, pin_memory=True)\n        return pin_tensor.to(self.device, non_blocking=True)\n\n    def _resolve_verify_layout(\n        self,\n        forward_batch: ForwardBatch,\n        bs: int,\n    ) -> Optional[RaggedVerifyLayout]:\n        layout = resolve_ragged_verify_layout(forward_batch)\n        if layout is None:\n            return None\n        if read_ragged_verify_mode() is not RaggedVerifyMode.COMPACT:\n            return None\n        if get_parallel().attn_cp_size > 1:\n            raise NotImplementedError(\n                \"DSV4 ragged verify does not support context parallel (CP); \"\n                \"set SGLANG_RAGGED_VERIFY_MODE off for CP runs.\"\n            )\n        if self.online_c128_mtp.enabled():\n            raise NotImplementedError(\n                \"DSV4 ragged verify does not support online c128 MTP; \"\n                \"set SGLANG_RAGGED_VERIFY_MODE off or disable online compress.\"\n            )\n        # Layout invariants (verify_lens >= 1, total == sum) are enforced in\n        # RaggedVerifyLayout.__post_init__; don't re-check the device tensor\n        # here -- that would D2H-sync the host-free verify prep path.\n        layout = layout.padded_to_bucket(padded_bs=bs)\n        return layout\n\n    def _target_verify_graph_key(\n        self,\n        bs: int,\n        ragged_layout: Optional[RaggedVerifyLayout],","sourceCodeStart":624,"sourceCodeEnd":660,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/attention/deepseek_v4_backend.py#L624-L660","documentation":"The DSV4 ragged-verify layout (enabled when SGLANG_RAGGED_VERIFY_MODE is COMPACT) reorganizes speculative verify tokens but has no support for attention context parallelism across ranks. The backend raises during metadata resolution (init_forward_metadata_out_graph / _build_forward_metadata) rather than producing incorrect cross-rank attention.","triggerScenarios":"Running with attn_cp_size > 1 while read_ragged_verify_mode() returns RaggedVerifyMode.COMPACT and a ragged verify layout resolves for the forward batch.","commonSituations":"Exporting SGLANG_RAGGED_VERIFY_MODE=compact for verify-path speed while enabling CP (e.g. --cp-size / attn CP) on a multi-GPU DeepSeek-V4 deployment; env vars left set from single-GPU experiments.","solutions":["Unset/disable SGLANG_RAGGED_VERIFY_MODE (set it off) for CP runs","Disable attention context parallelism if ragged verify mode is required"],"exampleFix":"# before\nSGLANG_RAGGED_VERIFY_MODE=compact sglang_launch ... --cp-size 4\n# after\nunset SGLANG_RAGGED_VERIFY_MODE; sglang_launch ... --cp-size 4","handlingStrategy":"validation","validationCode":"import os\nfrom sglang.srt.distributed import get_parallel\nif get_parallel().attn_cp_size > 1 and os.environ.get(\"SGLANG_RAGGED_VERIFY_MODE\", \"\").lower() == \"compact\":\n    del os.environ[\"SGLANG_RAGGED_VERIFY_MODE\"]  # incompatible with CP","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Centralize SGLANG_* env var review when changing parallelism settings","Run a short smoke launch when toggling CP size with experimental env vars set"],"tags":["deepseek-v4","ragged-verify","context-parallelism","env-var","speculative-decoding","sglang"],"backgroundTag":"unsupported-feature-combination","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}