sgl-project/sglang · error · ValueError

recent_window_tokens must be >= 0 or None

Error message

recent_window_tokens must be >= 0 or None

What it means

_reconstruct (rebuilding a dense [sink | rolled-recent] view from the packed cache) validates recent_window_tokens: if not None it must be >= 0. Negative windows are meaningless for computing tail_start.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/kvcache/qvg_packed_cache.py:347

            local_start_index=0,
            local_end_index=num_new,
            visible_local_end=min(self._global_end, self.cache_size),
            visible_global_end=self._global_end,
        )

    def _reconstruct(
        self,
        current_chunk_start: int,
        recent_window_tokens: int | None,
    ) -> tuple[torch.Tensor, torch.Tensor]:
        """Dense visible window = sink prefix ++ rolling recent tail, matching
        the dense cache's [sink | rolled-recent] buffer content."""
        sink_end = self._sink_end()
        if recent_window_tokens is None:
            tail_start = self._tail_start()
        else:
            if recent_window_tokens < 0:
                raise ValueError("recent_window_tokens must be >= 0 or None")
            tail_start = max(sink_end, current_chunk_start - recent_window_tokens)

        if tail_start <= sink_end:
            ranges = [(0, self._global_end)]
        else:
            ranges = [(0, sink_end), (tail_start, self._global_end)]

        visible_segments: list[tuple[_Segment, int, int]] = []
        visible_tokens = 0
        for g_lo, g_hi in ranges:
            for seg in self._all_segments():
                a = max(g_lo, seg.g0)
                b = min(g_hi, seg.g1)
                if b <= a:
                    continue
                visible_segments.append((seg, a - seg.g0, b - seg.g0))
                visible_tokens += b - a

View on GitHub (pinned to 0132848349)

Solutions

1. Use None for unlimited, 0 for sink-only
2. Clamp: `rw = None if rw is None else max(0, rw)` before the call
3. Audit config parsing that maps -1 to the window argument

Example fix

# before
cache.update_and_get_attention_kv(k, v, recent_window_tokens=-1)
# after
cache.update_and_get_attention_kv(k, v, recent_window_tokens=None)

rw = None if recent_window_tokens is None else max(0, int(recent_window_tokens))

def is_valid_window(w: int | None) -> bool:\n    return w is None or (isinstance(w, int) and w >= 0)

When it happens

Trigger: update_and_get_attention_kv on the packed cache with recent_window_tokens < 0 (e.g. -1 sentinel) reaching _reconstruct.

Common situations: Config code using -1 to mean 'disabled' colliding with this API's None-for-unlimited convention; arithmetic producing a negative window after subtraction of sink tokens.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• recent_window_tokens must be non-negative or None
• kv-canary: launch_canary_plan_kernels requires full_to_swa_i
• kv-canary: SWA slot {slot} is outside full_to_swa_index_mapp
• {debug_name}: cache_head_start required for head slice
• WindowedAttentionKVCache holds only the trailing window and

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/154617ad323b4ce8. Report an issue: GitHub.