{"record":{"id":"78b0189c44bd21f5","repo":"sgl-project/sglang","slug":"total-verify-tokens-total-verify-tokens-sum-v","errorCode":null,"errorMessage":"total_verify_tokens {total_verify_tokens} != sum(verify_lens_cpu) {sum(verify_lens_cpu)}","messagePattern":"total_verify_tokens (.+?) != sum\\(verify_lens_cpu\\) (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/speculative/ragged_verify.py","lineNumber":78,"sourceCode":"    cap: Optional[int] = None\n\n    def __post_init__(self) -> None:\n        if self.verify_lens_cpu is None:\n            return\n        if not self.verify_lens_cpu:\n            raise ValueError(\"RaggedVerifyLayout requires at least one request\")\n        if min(self.verify_lens_cpu) < 1:\n            raise ValueError(\n                f\"every request must verify the anchor (verify_len >= 1), got \"\n                f\"{self.verify_lens_cpu}\"\n            )\n        if self.cap is not None and max(self.verify_lens_cpu) > self.cap:\n            raise ValueError(\n                f\"capped layout has a row exceeding cap={self.cap}: \"\n                f\"{self.verify_lens_cpu}\"\n            )\n        if self.total_verify_tokens != sum(self.verify_lens_cpu):\n            raise ValueError(\n                f\"total_verify_tokens {self.total_verify_tokens} != \"\n                f\"sum(verify_lens_cpu) {sum(self.verify_lens_cpu)}\"\n            )\n        if not (self.total_verify_tokens <= self.graph_num_tokens):\n            raise ValueError(\n                f\"total_verify_tokens {self.total_verify_tokens} exceeds \"\n                f\"graph_num_tokens {self.graph_num_tokens}\"\n            )\n\n    @property\n    def bs(self) -> int:\n        return int(self.verify_lens.shape[0])\n\n    @classmethod\n    def _assemble_device(\n        cls,\n        *,\n        verify_lens: torch.Tensor,","sourceCodeStart":60,"sourceCodeEnd":96,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/speculative/ragged_verify.py#L60-L96","documentation":"RaggedVerifyLayout cross-checks that total_verify_tokens equals sum(verify_lens_cpu). These redundant fields must agree; a mismatch means the caller computed them inconsistently, indicating a bookkeeping bug in the batching code.","triggerScenarios":"Constructing the layout with total_verify_tokens set independently (e.g. a padded graph token count) rather than the exact sum of the row lengths.","commonSituations":"Refactoring that passes the CUDA-graph padded token count as total_verify_tokens instead of the true ragged total; stale field after editing verify_lens.","solutions":["Always pass total_verify_tokens=sum(verify_lens_cpu)","Derive it at the call site: RaggedVerifyLayout(..., total_verify_tokens=sum(lens))","Add a unit test asserting the invariant for your layout builder"],"exampleFix":"// before\nlayout = RaggedVerifyLayout(verify_lens_cpu=lens, total_verify_tokens=graph_num_tokens, ...)\n// after\nlayout = RaggedVerifyLayout(verify_lens_cpu=lens, total_verify_tokens=sum(lens), graph_num_tokens=graph_num_tokens, ...)","handlingStrategy":"validation","validationCode":"layout = RaggedVerifyLayout(verify_lens_cpu=lens, total_verify_tokens=sum(lens), graph_num_tokens=graph_num_tokens)","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Never set total_verify_tokens independently; always derive it as sum(verify_lens_cpu)","Add a unit test asserting layout invariants after any batching refactor"],"tags":["speculative-decoding","validation","invariant"],"backgroundTag":"input-validation-failed","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}