{"record":{"id":"ce31b31dd9a787ba","repo":"sgl-project/sglang","slug":"dspark-aux-hidden-capture-requires-pp-1","errorCode":null,"errorMessage":"DSPARK aux hidden capture requires PP=1.","messagePattern":"DSPARK aux hidden capture requires PP=1\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/models/kimi_k3.py","lineNumber":2888,"sourceCode":"                config.hidden_size,\n                quant_config=quant_config,\n                prefix=maybe_prefix(prefix, \"lm_head\"),\n                use_attn_tp_group=get_parallel().enable_dp_lm_head,\n            )\n        else:\n            self.lm_head = PPMissingLayer()\n        logit_scale = getattr(config, \"logit_scale\", 1.0)\n        self.logits_processor = LogitsProcessor(config=config, logit_scale=logit_scale)\n        self.capture_aux_hidden_states = False\n\n    def get_input_embeddings(self):\n        return self.model.embed_tokens\n\n    def set_dspark_layers_to_capture(self, layer_ids: list[int]) -> None:\n        if self.pp_group.world_size > 1:\n            # Capture layers living on non-last PP ranks would be silently\n            # skipped (the flag is only set on the last rank).\n            raise NotImplementedError(\"DSPARK aux hidden capture requires PP=1.\")\n        if not self.pp_group.is_last_rank:\n            return\n        if layer_ids is None:\n            raise ValueError(\n                \"DSPARK requires explicit layer_ids for aux hidden capture.\"\n            )\n        self.capture_aux_hidden_states = True\n        self.model.dspark_layers_to_capture = list(layer_ids)\n\n    @torch.no_grad()\n    def forward(\n        self,\n        input_ids: torch.Tensor,\n        positions: torch.Tensor,\n        forward_batch: ForwardBatch,\n        input_embeds: Optional[torch.Tensor] = None,\n        inputs_embeds: Optional[torch.Tensor] = None,\n        pp_proxy_tensors: Optional[PPProxyTensors] = None,","sourceCodeStart":2870,"sourceCodeEnd":2906,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/models/kimi_k3.py#L2870-L2906","documentation":"KimiK3.set_dspark_layers_to_capture enables DSPARK aux-hidden-state capture on specific layers by setting a flag that only takes effect on the last pipeline-parallel rank. With PP>1, requested layers on earlier ranks would be silently skipped, so the code raises NotImplementedError to force PP=1 rather than producing a broken capture.","triggerScenarios":"Calling set_dspark_layers_to_capture(layer_ids) (done by the DSPARK/speculative runtime during draft setup) while pp_group.world_size > 1 — i.e. launching the server with --pipeline-parallel-size > 1 together with DSPARK aux hidden capture.","commonSituations":"Enabling DSPARK speculative training/serving on a multi-node or PP-sharded Kimi K3 deployment; reusing PP flags from a non-DSPARK launch script when turning on DSPARK.","solutions":["Relaunch with --pipeline-parallel-size 1 (tensor parallel / expert parallel are fine).","If PP is required, disable the DSPARK aux-hidden capture feature.","Track upstream support for PP>1 capture (flag propagation across ranks) before retrying."],"exampleFix":"# before\npython -m sglang.launch_server --model kimi-k3 --pp 2 --speculative-algorithm DSPARK ...\n\n# after\npython -m sglang.launch_server --model kimi-k3 --pp 1 --tp 8 --speculative-algorithm DSPARK ...","handlingStrategy":"validation","validationCode":"from sglang.srt.distributed import get_pp_group\nif get_pp_group().world_size > 1:\n    raise RuntimeError(\"disable DSPARK aux capture or set --pp 1\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Keep PP=1 when enabling DSPARK on Kimi K3.","Document parallelism constraints in deployment scripts."],"tags":["dspark","pipeline-parallel","speculative-decoding","kimi"],"backgroundTag":"feature-parallelism-conflict","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}