sgl-project/sglang · error · ValueError

total_verify_tokens {total_verify_tokens} != sum(verify_lens

Error message

total_verify_tokens {total_verify_tokens} != sum(verify_lens_cpu) {sum(verify_lens_cpu)}

What it means

RaggedVerifyLayout cross-checks that total_verify_tokens equals sum(verify_lens_cpu). These redundant fields must agree; a mismatch means the caller computed them inconsistently, indicating a bookkeeping bug in the batching code.

Source

Thrown at python/sglang/srt/speculative/ragged_verify.py:78

    cap: Optional[int] = None

    def __post_init__(self) -> None:
        if self.verify_lens_cpu is None:
            return
        if not self.verify_lens_cpu:
            raise ValueError("RaggedVerifyLayout requires at least one request")
        if min(self.verify_lens_cpu) < 1:
            raise ValueError(
                f"every request must verify the anchor (verify_len >= 1), got "
                f"{self.verify_lens_cpu}"
            )
        if self.cap is not None and max(self.verify_lens_cpu) > self.cap:
            raise ValueError(
                f"capped layout has a row exceeding cap={self.cap}: "
                f"{self.verify_lens_cpu}"
            )
        if self.total_verify_tokens != sum(self.verify_lens_cpu):
            raise ValueError(
                f"total_verify_tokens {self.total_verify_tokens} != "
                f"sum(verify_lens_cpu) {sum(self.verify_lens_cpu)}"
            )
        if not (self.total_verify_tokens <= self.graph_num_tokens):
            raise ValueError(
                f"total_verify_tokens {self.total_verify_tokens} exceeds "
                f"graph_num_tokens {self.graph_num_tokens}"
            )

    @property
    def bs(self) -> int:
        return int(self.verify_lens.shape[0])

    @classmethod
    def _assemble_device(
        cls,
        *,
        verify_lens: torch.Tensor,

View on GitHub (pinned to 0132848349)

Solutions

1. Always pass total_verify_tokens=sum(verify_lens_cpu)
2. Derive it at the call site: RaggedVerifyLayout(..., total_verify_tokens=sum(lens))
3. Add a unit test asserting the invariant for your layout builder

Example fix

// before
layout = RaggedVerifyLayout(verify_lens_cpu=lens, total_verify_tokens=graph_num_tokens, ...)
// after
layout = RaggedVerifyLayout(verify_lens_cpu=lens, total_verify_tokens=sum(lens), graph_num_tokens=graph_num_tokens, ...)

layout = RaggedVerifyLayout(verify_lens_cpu=lens, total_verify_tokens=sum(lens), graph_num_tokens=graph_num_tokens)

When it happens

Trigger: Constructing the layout with total_verify_tokens set independently (e.g. a padded graph token count) rather than the exact sum of the row lengths.

Common situations: Refactoring that passes the CUDA-graph padded token count as total_verify_tokens instead of the true ragged total; stale field after editing verify_lens.

Related errors

• kv-canary: launch_canary_plan_kernels_torch_reference verify
• --speculative-ngram-external-sam-budget must be positive whe
• --speculative-ngram-external-corpus-max-tokens must be posit
• speculative_ngram_external_sam_budget must be less than or e
• kv-canary: speculative_num_draft_tokens must be non-negative

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/78b0189c44bd21f5. Report an issue: GitHub.