sgl-project/sglang · error · ValueError

RaggedVerifyLayout requires at least one request

Error message

RaggedVerifyLayout requires at least one request

What it means

RaggedVerifyLayout is a dataclass describing the ragged (per-request variable-length) verification layout for speculative decoding. Its __post_init__ validates the layout; an empty verify_lens_cpu (an empty list, not None) means zero requests in the verify batch, which is nonsensical for a verify forward pass, so it raises ValueError.

Source

Thrown at python/sglang/srt/speculative/ragged_verify.py:66

    graph_num_tokens: int
    extend_start_loc: torch.Tensor
    qo_indptr_device: torch.Tensor
    verify_lens_cpu: Optional[list[int]] = None
    total_verify_tokens: Optional[int] = None
    qo_indptr_host: Optional[torch.Tensor] = None
    kv_indptr_host: Optional[torch.Tensor] = None
    kv_lens_host: Optional[torch.Tensor] = None
    max_q_len: Optional[int] = None
    max_kv_len: Optional[int] = None
    # Per-row upper bound (capped padded variant); rows never exceed it, so
    # dense [bs, cap] consumers stay in bounds. None = full-coverage variant.
    cap: Optional[int] = None

    def __post_init__(self) -> None:
        if self.verify_lens_cpu is None:
            return
        if not self.verify_lens_cpu:
            raise ValueError("RaggedVerifyLayout requires at least one request")
        if min(self.verify_lens_cpu) < 1:
            raise ValueError(
                f"every request must verify the anchor (verify_len >= 1), got "
                f"{self.verify_lens_cpu}"
            )
        if self.cap is not None and max(self.verify_lens_cpu) > self.cap:
            raise ValueError(
                f"capped layout has a row exceeding cap={self.cap}: "
                f"{self.verify_lens_cpu}"
            )
        if self.total_verify_tokens != sum(self.verify_lens_cpu):
            raise ValueError(
                f"total_verify_tokens {self.total_verify_tokens} != "
                f"sum(verify_lens_cpu) {sum(self.verify_lens_cpu)}"
            )
        if not (self.total_verify_tokens <= self.graph_num_tokens):
            raise ValueError(
                f"total_verify_tokens {self.total_verify_tokens} exceeds "

View on GitHub (pinned to 0132848349)

Solutions

1. Ensure the verify path is only entered with a non-empty batch; skip verify when len(reqs)==0
2. Pass verify_lens_cpu=None if constructing a placeholder layout
3. If writing tests, use at least one request, e.g. [1] for a one-token anchor row

Example fix

// before
layout = RaggedVerifyLayout(verify_lens_cpu=[], total_verify_tokens=0, graph_num_tokens=64)
// after
layout = RaggedVerifyLayout(verify_lens_cpu=None)  # placeholder
# or skip verify entirely when the batch is empty

if not verify_lens_cpu:
    return  # nothing to verify; skip building the layout
layout = RaggedVerifyLayout(verify_lens_cpu=verify_lens_cpu, ...)

def is_nonempty_verify_batch(lens: list[int] | None) -> bool:
    return lens is None or len(lens) > 0

When it happens

Trigger: Constructing RaggedVerifyLayout(verify_lens_cpu=[]) or passing an empty running-batch schedule batch that produces an empty list. A None value is allowed (lazy/unset layout), but an empty sequence is not.

Common situations: Calling target-verify with an empty req list, e.g. after all requests finish mid-overlap-scheduling, or building the layout from a filtered batch that ends up empty in tests.

Related errors

• kv-canary: launch_canary_plan_kernels_torch_reference verify
• --speculative-ngram-external-sam-budget must be positive whe
• --speculative-ngram-external-corpus-max-tokens must be posit
• speculative_ngram_external_sam_budget must be less than or e
• kv-canary: speculative_num_draft_tokens must be non-negative

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/9912b6775a69c8b9. Report an issue: GitHub.