sgl-project/sglang · error · ValueError

capped layout has a row exceeding cap={self.cap}: {verify_le

Error message

capped layout has a row exceeding cap={self.cap}: {verify_lens_cpu}

What it means

When RaggedVerifyLayout has a cap (max tokens per ragged row, e.g. the CUDA-graph capture window num_draft_tokens+1), no single request's verify_len may exceed it, otherwise the row cannot be represented by the captured graphs.

Source

Thrown at python/sglang/srt/speculative/ragged_verify.py:73

    kv_lens_host: Optional[torch.Tensor] = None
    max_q_len: Optional[int] = None
    max_kv_len: Optional[int] = None
    # Per-row upper bound (capped padded variant); rows never exceed it, so
    # dense [bs, cap] consumers stay in bounds. None = full-coverage variant.
    cap: Optional[int] = None

    def __post_init__(self) -> None:
        if self.verify_lens_cpu is None:
            return
        if not self.verify_lens_cpu:
            raise ValueError("RaggedVerifyLayout requires at least one request")
        if min(self.verify_lens_cpu) < 1:
            raise ValueError(
                f"every request must verify the anchor (verify_len >= 1), got "
                f"{self.verify_lens_cpu}"
            )
        if self.cap is not None and max(self.verify_lens_cpu) > self.cap:
            raise ValueError(
                f"capped layout has a row exceeding cap={self.cap}: "
                f"{self.verify_lens_cpu}"
            )
        if self.total_verify_tokens != sum(self.verify_lens_cpu):
            raise ValueError(
                f"total_verify_tokens {self.total_verify_tokens} != "
                f"sum(verify_lens_cpu) {sum(self.verify_lens_cpu)}"
            )
        if not (self.total_verify_tokens <= self.graph_num_tokens):
            raise ValueError(
                f"total_verify_tokens {self.total_verify_tokens} exceeds "
                f"graph_num_tokens {self.graph_num_tokens}"
            )

    @property
    def bs(self) -> int:
        return int(self.verify_lens.shape[0])

View on GitHub (pinned to 0132848349)

Solutions

1. Ensure num_draft_tokens used to build the layout matches the algorithm config
2. Clamp verify_lens to cap or raise the cap
3. Regenerate capture layouts after changing spec-decoding config

Example fix

// before
layout = RaggedVerifyLayout(verify_lens_cpu=[9], cap=5, ...)
// after
verify_lens = [min(v, cap) for v in verify_lens]  # or increase cap to match max draft len

assert cap is None or max(verify_lens_cpu) <= cap, (cap, verify_lens_cpu)
verify_lens = [min(v, cap) for v in verify_lens_cpu]  # if clamping is semantically OK

def fits_cap(lens: list[int], cap: int | None) -> bool:
    return cap is None or max(lens) <= cap

When it happens

Trigger: Setting cap=N while some verify_len > N, e.g. a request accepting more draft tokens than speculative_num_draft_tokens, or mixing an uncapped layout into a capped capture path.

Common situations: Misconfigured speculative_num_draft_tokens vs actual draft lengths; changing the draft token count without rebuilding the capture layout.

Related errors

• kv-canary: launch_canary_plan_kernels_torch_reference verify
• --speculative-ngram-external-sam-budget must be positive whe
• --speculative-ngram-external-corpus-max-tokens must be posit
• speculative_ngram_external_sam_budget must be less than or e
• kv-canary: speculative_num_draft_tokens must be non-negative

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/f1a76ee9d52210b6. Report an issue: GitHub.