sgl-project/sglang · error · ValueError

capture layout needs 1 <= num_slots <= num_tokens, got num_s

Error message

capture layout needs 1 <= num_slots <= num_tokens, got num_slots={num_slots}, num_tokens={num_tokens}

What it means

build_capture_verify_lens builds the synthetic verify-length list used when capturing CUDA graphs for ragged verification. It requires at least one row (num_slots >= 1) and that num_tokens >= num_slots (each row needs at least one token — the anchor).

Source

Thrown at python/sglang/srt/speculative/ragged_verify.py:205

        if cap is not None:
            padded = torch.clamp(padded, max=cap)

        return RaggedVerifyLayout._assemble_device(
            verify_lens=padded,
            graph_num_tokens=self.graph_num_tokens,
            total_verify_tokens=None if cap is not None else self.graph_num_tokens,
            cap=cap,
        )


def build_capture_verify_lens(
    *,
    num_tokens: int,
    num_slots: int,
    num_draft_tokens: int,
) -> list[int]:
    if num_slots < 1 or num_tokens < num_slots:
        raise ValueError(
            f"capture layout needs 1 <= num_slots <= num_tokens, got "
            f"num_slots={num_slots}, num_tokens={num_tokens}"
        )
    if num_tokens > num_slots * num_draft_tokens:
        raise ValueError(
            f"capture layout cannot pack num_tokens={num_tokens} into "
            f"{num_slots} rows of at most {num_draft_tokens} tokens"
        )
    base = num_tokens // num_slots
    rem = num_tokens - base * num_slots
    return [base + 1] * rem + [base] * (num_slots - rem)


def resolve_ragged_verify_layout(forward_batch) -> Optional[RaggedVerifyLayout]:
    """Layout riding the batch's spec input, or None. Tolerates the runner's
    ad-hoc replay batch views, which may not carry spec_info at all."""
    spec_info = getattr(forward_batch, "spec_info", None)
    if spec_info is None:

View on GitHub (pinned to 0132848349)

Solutions

1. Ensure each capture tier has num_tokens >= num_slots (at least 1 token per row)
2. Fix tier definitions so token windows scale with slot counts
3. For degenerate tiers, skip capture instead of calling the builder

Example fix

// before
lens = build_capture_verify_lens(num_tokens=4, num_slots=8, num_draft_tokens=4)
// after
lens = build_capture_verify_lens(num_tokens=8, num_slots=4, num_draft_tokens=4)

assert 1 <= num_slots <= num_tokens, (num_slots, num_tokens)
lens = build_capture_verify_lens(num_tokens=num_tokens, num_slots=num_slots, num_draft_tokens=nd)

def valid_capture_inputs(num_tokens: int, num_slots: int) -> bool:
    return num_slots >= 1 and num_tokens >= num_slots

When it happens

Trigger: Calling build_capture_verify_lens(num_slots=0, ...) or with num_tokens < num_slots (e.g. 4 tokens into 8 slots), which happens when capture batch sizes are computed from an empty or undersized token budget.

Common situations: Misconfigured cuda-graph capture tiers (tier token window smaller than its slot count); tests exercising the capture layout builder with degenerate inputs.

Related errors

• total_verify_tokens {total_verify_tokens} exceeds graph_num_
• capture layout cannot pack num_tokens={num_tokens} into {num
• kv-canary: launch_canary_plan_kernels_torch_reference verify
• --speculative-ngram-external-sam-budget must be positive whe
• --speculative-ngram-external-corpus-max-tokens must be posit

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/bd5939d777affbf3. Report an issue: GitHub.