{"record":{"id":"f1a76ee9d52210b6","repo":"sgl-project/sglang","slug":"capped-layout-has-a-row-exceeding-cap-self-cap","errorCode":null,"errorMessage":"capped layout has a row exceeding cap={self.cap}: {verify_lens_cpu}","messagePattern":"capped layout has a row exceeding cap=(.+?): (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/speculative/ragged_verify.py","lineNumber":73,"sourceCode":"    kv_lens_host: Optional[torch.Tensor] = None\n    max_q_len: Optional[int] = None\n    max_kv_len: Optional[int] = None\n    # Per-row upper bound (capped padded variant); rows never exceed it, so\n    # dense [bs, cap] consumers stay in bounds. None = full-coverage variant.\n    cap: Optional[int] = None\n\n    def __post_init__(self) -> None:\n        if self.verify_lens_cpu is None:\n            return\n        if not self.verify_lens_cpu:\n            raise ValueError(\"RaggedVerifyLayout requires at least one request\")\n        if min(self.verify_lens_cpu) < 1:\n            raise ValueError(\n                f\"every request must verify the anchor (verify_len >= 1), got \"\n                f\"{self.verify_lens_cpu}\"\n            )\n        if self.cap is not None and max(self.verify_lens_cpu) > self.cap:\n            raise ValueError(\n                f\"capped layout has a row exceeding cap={self.cap}: \"\n                f\"{self.verify_lens_cpu}\"\n            )\n        if self.total_verify_tokens != sum(self.verify_lens_cpu):\n            raise ValueError(\n                f\"total_verify_tokens {self.total_verify_tokens} != \"\n                f\"sum(verify_lens_cpu) {sum(self.verify_lens_cpu)}\"\n            )\n        if not (self.total_verify_tokens <= self.graph_num_tokens):\n            raise ValueError(\n                f\"total_verify_tokens {self.total_verify_tokens} exceeds \"\n                f\"graph_num_tokens {self.graph_num_tokens}\"\n            )\n\n    @property\n    def bs(self) -> int:\n        return int(self.verify_lens.shape[0])\n","sourceCodeStart":55,"sourceCodeEnd":91,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/speculative/ragged_verify.py#L55-L91","documentation":"When RaggedVerifyLayout has a cap (max tokens per ragged row, e.g. the CUDA-graph capture window num_draft_tokens+1), no single request's verify_len may exceed it, otherwise the row cannot be represented by the captured graphs.","triggerScenarios":"Setting cap=N while some verify_len > N, e.g. a request accepting more draft tokens than speculative_num_draft_tokens, or mixing an uncapped layout into a capped capture path.","commonSituations":"Misconfigured speculative_num_draft_tokens vs actual draft lengths; changing the draft token count without rebuilding the capture layout.","solutions":["Ensure num_draft_tokens used to build the layout matches the algorithm config","Clamp verify_lens to cap or raise the cap","Regenerate capture layouts after changing spec-decoding config"],"exampleFix":"// before\nlayout = RaggedVerifyLayout(verify_lens_cpu=[9], cap=5, ...)\n// after\nverify_lens = [min(v, cap) for v in verify_lens]  # or increase cap to match max draft len","handlingStrategy":"validation","validationCode":"assert cap is None or max(verify_lens_cpu) <= cap, (cap, verify_lens_cpu)\nverify_lens = [min(v, cap) for v in verify_lens_cpu]  # if clamping is semantically OK","typeGuard":"def fits_cap(lens: list[int], cap: int | None) -> bool:\n    return cap is None or max(lens) <= cap","tryCatchPattern":null,"preventionTips":["Keep num_draft_tokens consistent between layout construction and config","Clamp accepted lengths to the draft window before building the layout"],"tags":["speculative-decoding","validation","capacity-limit"],"backgroundTag":"input-validation-failed","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}