sgl-project/sglang · error · ValueError

capture layout cannot pack num_tokens={num_tokens} into {num

Error message

capture layout cannot pack num_tokens={num_tokens} into {num_slots} rows of at most {num_draft_tokens} tokens

What it means

Each capture row can hold at most num_draft_tokens tokens (the draft window size). If num_tokens > num_slots * num_draft_tokens, no legal packing exists, so build_capture_verify_lens raises ValueError.

Source

Thrown at python/sglang/srt/speculative/ragged_verify.py:210

            graph_num_tokens=self.graph_num_tokens,
            total_verify_tokens=None if cap is not None else self.graph_num_tokens,
            cap=cap,
        )


def build_capture_verify_lens(
    *,
    num_tokens: int,
    num_slots: int,
    num_draft_tokens: int,
) -> list[int]:
    if num_slots < 1 or num_tokens < num_slots:
        raise ValueError(
            f"capture layout needs 1 <= num_slots <= num_tokens, got "
            f"num_slots={num_slots}, num_tokens={num_tokens}"
        )
    if num_tokens > num_slots * num_draft_tokens:
        raise ValueError(
            f"capture layout cannot pack num_tokens={num_tokens} into "
            f"{num_slots} rows of at most {num_draft_tokens} tokens"
        )
    base = num_tokens // num_slots
    rem = num_tokens - base * num_slots
    return [base + 1] * rem + [base] * (num_slots - rem)


def resolve_ragged_verify_layout(forward_batch) -> Optional[RaggedVerifyLayout]:
    """Layout riding the batch's spec input, or None. Tolerates the runner's
    ad-hoc replay batch views, which may not carry spec_info at all."""
    spec_info = getattr(forward_batch, "spec_info", None)
    if spec_info is None:
        return None
    return spec_info.ragged_verify_layout


class RaggedTargetVerifyGeometry(msgspec.Struct):

View on GitHub (pinned to 0132848349)

Solutions

1. Increase num_slots so num_slots * num_draft_tokens >= num_tokens
2. Reduce the tier's num_tokens
3. Recompute tiers whenever num_draft_tokens changes

Example fix

// before
lens = build_capture_verify_lens(num_tokens=32, num_slots=4, num_draft_tokens=4)
// after
lens = build_capture_verify_lens(num_tokens=32, num_slots=8, num_draft_tokens=4)

assert num_tokens <= num_slots * num_draft_tokens
lens = build_capture_verify_lens(...)

def packable(num_tokens: int, num_slots: int, num_draft_tokens: int) -> bool:
    return num_tokens <= num_slots * num_draft_tokens

When it happens

Trigger: Calling the builder with a token budget exceeding slots times per-row capacity, e.g. num_tokens=32, num_slots=4, num_draft_tokens=4 (max 16).

Common situations: Tier windows that don't respect the (num_draft_tokens+1 or num_draft_tokens) row capacity after changing speculative_num_draft_tokens; overpacked-tier tests (test_rejects_overpacked_tier).

Related errors

• total_verify_tokens {total_verify_tokens} exceeds graph_num_
• capture layout needs 1 <= num_slots <= num_tokens, got num_s
• kv-canary: launch_canary_plan_kernels_torch_reference verify
• --speculative-ngram-external-sam-budget must be positive whe
• --speculative-ngram-external-corpus-max-tokens must be posit

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/e5f68152cc169417. Report an issue: GitHub.