{"record":{"id":"bd5939d777affbf3","repo":"sgl-project/sglang","slug":"capture-layout-needs-1-num-slots-num-tokens","errorCode":null,"errorMessage":"capture layout needs 1 <= num_slots <= num_tokens, got num_slots={num_slots}, num_tokens={num_tokens}","messagePattern":"capture layout needs 1 <= num_slots <= num_tokens, got num_slots=(.+?), num_tokens=(.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/speculative/ragged_verify.py","lineNumber":205,"sourceCode":"        if cap is not None:\n            padded = torch.clamp(padded, max=cap)\n\n        return RaggedVerifyLayout._assemble_device(\n            verify_lens=padded,\n            graph_num_tokens=self.graph_num_tokens,\n            total_verify_tokens=None if cap is not None else self.graph_num_tokens,\n            cap=cap,\n        )\n\n\ndef build_capture_verify_lens(\n    *,\n    num_tokens: int,\n    num_slots: int,\n    num_draft_tokens: int,\n) -> list[int]:\n    if num_slots < 1 or num_tokens < num_slots:\n        raise ValueError(\n            f\"capture layout needs 1 <= num_slots <= num_tokens, got \"\n            f\"num_slots={num_slots}, num_tokens={num_tokens}\"\n        )\n    if num_tokens > num_slots * num_draft_tokens:\n        raise ValueError(\n            f\"capture layout cannot pack num_tokens={num_tokens} into \"\n            f\"{num_slots} rows of at most {num_draft_tokens} tokens\"\n        )\n    base = num_tokens // num_slots\n    rem = num_tokens - base * num_slots\n    return [base + 1] * rem + [base] * (num_slots - rem)\n\n\ndef resolve_ragged_verify_layout(forward_batch) -> Optional[RaggedVerifyLayout]:\n    \"\"\"Layout riding the batch's spec input, or None. Tolerates the runner's\n    ad-hoc replay batch views, which may not carry spec_info at all.\"\"\"\n    spec_info = getattr(forward_batch, \"spec_info\", None)\n    if spec_info is None:","sourceCodeStart":187,"sourceCodeEnd":223,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/speculative/ragged_verify.py#L187-L223","documentation":"build_capture_verify_lens builds the synthetic verify-length list used when capturing CUDA graphs for ragged verification. It requires at least one row (num_slots >= 1) and that num_tokens >= num_slots (each row needs at least one token — the anchor).","triggerScenarios":"Calling build_capture_verify_lens(num_slots=0, ...) or with num_tokens < num_slots (e.g. 4 tokens into 8 slots), which happens when capture batch sizes are computed from an empty or undersized token budget.","commonSituations":"Misconfigured cuda-graph capture tiers (tier token window smaller than its slot count); tests exercising the capture layout builder with degenerate inputs.","solutions":["Ensure each capture tier has num_tokens >= num_slots (at least 1 token per row)","Fix tier definitions so token windows scale with slot counts","For degenerate tiers, skip capture instead of calling the builder"],"exampleFix":"// before\nlens = build_capture_verify_lens(num_tokens=4, num_slots=8, num_draft_tokens=4)\n// after\nlens = build_capture_verify_lens(num_tokens=8, num_slots=4, num_draft_tokens=4)","handlingStrategy":"validation","validationCode":"assert 1 <= num_slots <= num_tokens, (num_slots, num_tokens)\nlens = build_capture_verify_lens(num_tokens=num_tokens, num_slots=num_slots, num_draft_tokens=nd)","typeGuard":"def valid_capture_inputs(num_tokens: int, num_slots: int) -> bool:\n    return num_slots >= 1 and num_tokens >= num_slots","tryCatchPattern":null,"preventionTips":["Design capture tiers so each tier's token window >= its slot count","Skip degenerate tiers rather than calling the builder with them"],"tags":["speculative-decoding","cuda-graph","validation"],"backgroundTag":"input-validation-failed","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}