sgl-project/sglang · error · ValueError

{start_len=} must be non-negative

Error message

{start_len=} must be non-negative

What it means

StateCapturer.get_topk slices req_to_token from start_len to seqlen-1 to gather routed-expert/indexer top-k state; a negative start_len would slice invalid memory, so it raises ValueError immediately.

Source

Thrown at python/sglang/srt/state_capturer/base.py:155

        Default assumes per-rank-local capture: each rank writes [:local_num_tokens)
        to its own device_cache. Subclasses with global-tensor capture semantics
        (e.g. shared cuda graph buffer indexed by dp_rank) should override and
        consume can_run_graph / cuda_graph_batch.
        """
        del can_run_graph, cuda_graph_batch  # reserved for subclass override
        num_tokens = forward_batch.out_cache_loc.shape[0]
        return self.device_cache.buffer[:num_tokens, :, : self.topk_size]

    def get_topk(
        self,
        req_pool_idx: int,
        seqlen: int,
        req_to_token_pool: ReqToTokenPool,
        start_len: int = 0,
    ) -> torch.Tensor:
        if start_len < 0:
            raise ValueError(f"{start_len=} must be non-negative")
        start_len = min(start_len, seqlen - 1)
        cache_pool_idx = (
            req_to_token_pool.req_to_token[req_pool_idx][start_len : seqlen - 1]
            .cpu()
            .clone()
        )
        return self.host_cache.buffer[cache_pool_idx]

    def on_forward_end(
        self,
        forward_batch: ForwardBatch,
        can_run_graph: bool,
        cuda_graph_batch: Optional[int],
        no_copy_to_cpu: bool = False,
    ) -> Optional[TopkCaptureOutput]:
        """If no_copy_to_cpu is True, return a TopkCaptureOutput holding GPU tensors so
        the overlap thread can do non-blocking D2H + finalize itself. Otherwise sync
        D2H inline and return None (legacy non-overlap path).

View on GitHub (pinned to 0132848349)

Solutions

1. Clamp: start_len = max(0, min(start_len, seqlen - 1)) before calling
2. Fix the caller's window arithmetic (use max(0, seqlen - window))
3. Skip collection for sequences where the computed start is negative

Example fix

# before
state.get_topk(req_pool_idx, seqlen, pool, start_len=seqlen - window)
# after
state.get_topk(req_pool_idx, seqlen, pool, start_len=max(0, seqlen - window))

start_len = max(0, min(start_len, seqlen - 1))
topk = capturer.get_topk(req_pool_idx, seqlen, pool, start_len=start_len)

When it happens

Trigger: Calling get_topk(..., start_len=-1) or with a computed start_len that went negative, e.g. seqlen - context_window underflow for a short sequence, from _maybe_collect_routed_experts / _maybe_collect_indexer_topk.

Common situations: Sliding-window or context-window start computation underflowing for sequences shorter than the window; off-by-one in capture-range arithmetic.

Related errors

• store_kv: token_ids has {n} entries but kv_indices has {len(
• every request must verify the anchor (verify_len >= 1), got
• v_cache must be provided
• q can only be None when only_qv=True
• q must be provided unless qv is provided with only_qv=True

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/b4d05a3f6d54851d. Report an issue: GitHub.