{"record":{"id":"88fb0c29bedb9981","repo":"sgl-project/sglang","slug":"every-request-must-verify-the-anchor-verify-len","errorCode":null,"errorMessage":"every request must verify the anchor (verify_len >= 1), got {verify_lens_cpu}","messagePattern":"every request must verify the anchor \\(verify_len >= 1\\), got (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/speculative/ragged_verify.py","lineNumber":68,"sourceCode":"    qo_indptr_device: torch.Tensor\n    verify_lens_cpu: Optional[list[int]] = None\n    total_verify_tokens: Optional[int] = None\n    qo_indptr_host: Optional[torch.Tensor] = None\n    kv_indptr_host: Optional[torch.Tensor] = None\n    kv_lens_host: Optional[torch.Tensor] = None\n    max_q_len: Optional[int] = None\n    max_kv_len: Optional[int] = None\n    # Per-row upper bound (capped padded variant); rows never exceed it, so\n    # dense [bs, cap] consumers stay in bounds. None = full-coverage variant.\n    cap: Optional[int] = None\n\n    def __post_init__(self) -> None:\n        if self.verify_lens_cpu is None:\n            return\n        if not self.verify_lens_cpu:\n            raise ValueError(\"RaggedVerifyLayout requires at least one request\")\n        if min(self.verify_lens_cpu) < 1:\n            raise ValueError(\n                f\"every request must verify the anchor (verify_len >= 1), got \"\n                f\"{self.verify_lens_cpu}\"\n            )\n        if self.cap is not None and max(self.verify_lens_cpu) > self.cap:\n            raise ValueError(\n                f\"capped layout has a row exceeding cap={self.cap}: \"\n                f\"{self.verify_lens_cpu}\"\n            )\n        if self.total_verify_tokens != sum(self.verify_lens_cpu):\n            raise ValueError(\n                f\"total_verify_tokens {self.total_verify_tokens} != \"\n                f\"sum(verify_lens_cpu) {sum(self.verify_lens_cpu)}\"\n            )\n        if not (self.total_verify_tokens <= self.graph_num_tokens):\n            raise ValueError(\n                f\"total_verify_tokens {self.total_verify_tokens} exceeds \"\n                f\"graph_num_tokens {self.graph_num_tokens}\"\n            )","sourceCodeStart":50,"sourceCodeEnd":86,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/speculative/ragged_verify.py#L50-L86","documentation":"RaggedVerifyLayout requires every request's verify length to be >= 1 because the first token of each row is the anchor (the accepted prefix token) that must always be verified. A verify_len of 0 would mean the request verifies nothing, breaking ragged attention indexing.","triggerScenarios":"Passing verify_lens_cpu containing a 0 entry, e.g. [3, 0, 2], typically from an off-by-one when computing accepted_len + 1 or a misconfigured draft length of 0 for some request.","commonSituations":"Off-by-one bugs in spec-decoding accept-length bookkeeping (using accept_len instead of accept_len+1), or a draft model returning zero draft tokens for a request.","solutions":["Compute verify_len as accepted_len + 1 (anchor included) per request","Clamp: verify_lens = [max(1, v) for v in verify_lens] if 0 can appear","Check the draft worker emits at least one token per request"],"exampleFix":"// before\nverify_lens = [req.accepted_len for req in reqs]\n// after\nverify_lens = [req.accepted_len + 1 for req in reqs]  # anchor token included","handlingStrategy":"validation","validationCode":"verify_lens = [v if v >= 1 else 1 for v in verify_lens]  # or fix upstream: accepted_len + 1","typeGuard":"def valid_verify_lens(lens: list[int]) -> bool:\n    return len(lens) > 0 and min(lens) >= 1","tryCatchPattern":null,"preventionTips":["Always compute verify_len as accepted_len + 1 (anchor included)","Assert min(verify_lens) >= 1 in scheduler debug logs when spec metrics look off"],"tags":["speculative-decoding","validation","off-by-one"],"backgroundTag":"input-validation-failed","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}