sgl-project/sglang · error · ValueError

every request must verify the anchor (verify_len >= 1), got

Error message

every request must verify the anchor (verify_len >= 1), got {verify_lens_cpu}

What it means

RaggedVerifyLayout requires every request's verify length to be >= 1 because the first token of each row is the anchor (the accepted prefix token) that must always be verified. A verify_len of 0 would mean the request verifies nothing, breaking ragged attention indexing.

Source

Thrown at python/sglang/srt/speculative/ragged_verify.py:68

    qo_indptr_device: torch.Tensor
    verify_lens_cpu: Optional[list[int]] = None
    total_verify_tokens: Optional[int] = None
    qo_indptr_host: Optional[torch.Tensor] = None
    kv_indptr_host: Optional[torch.Tensor] = None
    kv_lens_host: Optional[torch.Tensor] = None
    max_q_len: Optional[int] = None
    max_kv_len: Optional[int] = None
    # Per-row upper bound (capped padded variant); rows never exceed it, so
    # dense [bs, cap] consumers stay in bounds. None = full-coverage variant.
    cap: Optional[int] = None

    def __post_init__(self) -> None:
        if self.verify_lens_cpu is None:
            return
        if not self.verify_lens_cpu:
            raise ValueError("RaggedVerifyLayout requires at least one request")
        if min(self.verify_lens_cpu) < 1:
            raise ValueError(
                f"every request must verify the anchor (verify_len >= 1), got "
                f"{self.verify_lens_cpu}"
            )
        if self.cap is not None and max(self.verify_lens_cpu) > self.cap:
            raise ValueError(
                f"capped layout has a row exceeding cap={self.cap}: "
                f"{self.verify_lens_cpu}"
            )
        if self.total_verify_tokens != sum(self.verify_lens_cpu):
            raise ValueError(
                f"total_verify_tokens {self.total_verify_tokens} != "
                f"sum(verify_lens_cpu) {sum(self.verify_lens_cpu)}"
            )
        if not (self.total_verify_tokens <= self.graph_num_tokens):
            raise ValueError(
                f"total_verify_tokens {self.total_verify_tokens} exceeds "
                f"graph_num_tokens {self.graph_num_tokens}"
            )

View on GitHub (pinned to 0132848349)

Solutions

1. Compute verify_len as accepted_len + 1 (anchor included) per request
2. Clamp: verify_lens = [max(1, v) for v in verify_lens] if 0 can appear
3. Check the draft worker emits at least one token per request

Example fix

// before
verify_lens = [req.accepted_len for req in reqs]
// after
verify_lens = [req.accepted_len + 1 for req in reqs]  # anchor token included

verify_lens = [v if v >= 1 else 1 for v in verify_lens]  # or fix upstream: accepted_len + 1

def valid_verify_lens(lens: list[int]) -> bool:
    return len(lens) > 0 and min(lens) >= 1

When it happens

Trigger: Passing verify_lens_cpu containing a 0 entry, e.g. [3, 0, 2], typically from an off-by-one when computing accepted_len + 1 or a misconfigured draft length of 0 for some request.

Common situations: Off-by-one bugs in spec-decoding accept-length bookkeeping (using accept_len instead of accept_len+1), or a draft model returning zero draft tokens for a request.

Related errors

• kv-canary: launch_canary_plan_kernels_torch_reference verify
• --speculative-ngram-external-sam-budget must be positive whe
• --speculative-ngram-external-corpus-max-tokens must be posit
• speculative_ngram_external_sam_budget must be less than or e
• kv-canary: speculative_num_draft_tokens must be non-negative

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/88fb0c29bedb9981. Report an issue: GitHub.