{"record":{"id":"e5f68152cc169417","repo":"sgl-project/sglang","slug":"capture-layout-cannot-pack-num-tokens-num-tokens","errorCode":null,"errorMessage":"capture layout cannot pack num_tokens={num_tokens} into {num_slots} rows of at most {num_draft_tokens} tokens","messagePattern":"capture layout cannot pack num_tokens=(.+?) into (.+?) rows of at most (.+?) tokens","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/speculative/ragged_verify.py","lineNumber":210,"sourceCode":"            graph_num_tokens=self.graph_num_tokens,\n            total_verify_tokens=None if cap is not None else self.graph_num_tokens,\n            cap=cap,\n        )\n\n\ndef build_capture_verify_lens(\n    *,\n    num_tokens: int,\n    num_slots: int,\n    num_draft_tokens: int,\n) -> list[int]:\n    if num_slots < 1 or num_tokens < num_slots:\n        raise ValueError(\n            f\"capture layout needs 1 <= num_slots <= num_tokens, got \"\n            f\"num_slots={num_slots}, num_tokens={num_tokens}\"\n        )\n    if num_tokens > num_slots * num_draft_tokens:\n        raise ValueError(\n            f\"capture layout cannot pack num_tokens={num_tokens} into \"\n            f\"{num_slots} rows of at most {num_draft_tokens} tokens\"\n        )\n    base = num_tokens // num_slots\n    rem = num_tokens - base * num_slots\n    return [base + 1] * rem + [base] * (num_slots - rem)\n\n\ndef resolve_ragged_verify_layout(forward_batch) -> Optional[RaggedVerifyLayout]:\n    \"\"\"Layout riding the batch's spec input, or None. Tolerates the runner's\n    ad-hoc replay batch views, which may not carry spec_info at all.\"\"\"\n    spec_info = getattr(forward_batch, \"spec_info\", None)\n    if spec_info is None:\n        return None\n    return spec_info.ragged_verify_layout\n\n\nclass RaggedTargetVerifyGeometry(msgspec.Struct):","sourceCodeStart":192,"sourceCodeEnd":228,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/speculative/ragged_verify.py#L192-L228","documentation":"Each capture row can hold at most num_draft_tokens tokens (the draft window size). If num_tokens > num_slots * num_draft_tokens, no legal packing exists, so build_capture_verify_lens raises ValueError.","triggerScenarios":"Calling the builder with a token budget exceeding slots times per-row capacity, e.g. num_tokens=32, num_slots=4, num_draft_tokens=4 (max 16).","commonSituations":"Tier windows that don't respect the (num_draft_tokens+1 or num_draft_tokens) row capacity after changing speculative_num_draft_tokens; overpacked-tier tests (test_rejects_overpacked_tier).","solutions":["Increase num_slots so num_slots * num_draft_tokens >= num_tokens","Reduce the tier's num_tokens","Recompute tiers whenever num_draft_tokens changes"],"exampleFix":"// before\nlens = build_capture_verify_lens(num_tokens=32, num_slots=4, num_draft_tokens=4)\n// after\nlens = build_capture_verify_lens(num_tokens=32, num_slots=8, num_draft_tokens=4)","handlingStrategy":"validation","validationCode":"assert num_tokens <= num_slots * num_draft_tokens\nlens = build_capture_verify_lens(...)","typeGuard":"def packable(num_tokens: int, num_slots: int, num_draft_tokens: int) -> bool:\n    return num_tokens <= num_slots * num_draft_tokens","tryCatchPattern":null,"preventionTips":["Recompute capture tiers whenever num_draft_tokens changes","Keep tier token budgets <= slots * per-row capacity"],"tags":["speculative-decoding","cuda-graph","validation"],"backgroundTag":"capacity-exceeded","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}