sgl-project/sglang · critical · RuntimeError

Invalid {debug_name} write range: local=[{local_start_index}

Error message

Invalid {debug_name} write range: local=[{local_start_index}, {local_end_index}), global_end={global_end_index}, prev_local_end={local_end_index_prev}, kv_cache_size={kv_cache_size}, num_new_tokens={num_new_tokens}, current_start={current_chunk_start}, current_end={current_chunk_end}

What it means

A RuntimeError from update_and_get_attention_kv asserting the computed local write window is sane: start >= 0, end <= kv_cache_size, and the window length equals num_new_tokens. It fires when chunk bookkeeping (global_end, previous local end, current_chunk_start/end) is inconsistent with the token count being written.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/kvcache/causal_attention_cache.py:287

                if self._has_pinned_sink() and self.pinned_start >= sink_tokens:
                    self.pinned_start -= num_evicted_tokens

                # if we move the minimum number of tokens, the right bound of the append token would be aligned with end of the buffer
                local_end_index = kv_cache_size
            else:
                # enough space, directly append new tokens after end of previous chunk
                local_end_index = local_end_index_prev + appended_tokens
            local_start_index = local_end_index - num_new_tokens
            updated_local_end = local_end_index
            # after filling in the proceeded new chunk, the global end aligns with the global end of the current chunk
            updated_global_end = current_chunk_end

        if (
            local_start_index < 0
            or local_end_index > kv_cache_size
            or local_end_index - local_start_index != num_new_tokens
        ):
            raise RuntimeError(
                f"Invalid {debug_name} write range: "
                f"local=[{local_start_index}, {local_end_index}), "
                f"global_end={global_end_index}, "
                f"prev_local_end={local_end_index_prev}, "
                f"kv_cache_size={kv_cache_size}, "
                f"num_new_tokens={num_new_tokens}, "
                f"current_start={current_chunk_start}, current_end={current_chunk_end}"
            )

        if self.k.requires_grad:
            self.k = self.k.detach()
        if self.v.requires_grad:
            self.v = self.v.detach()
        attn_start_index = max(0, updated_local_end - self.attention_window_size)

        # write fresh kv and return visible view
        if cache_head_slice is None:
            self.k[:, local_start_index:local_end_index] = key

View on GitHub (pinned to 0132848349)

Solutions

1. Check the printed diagnostics: ensure current_chunk_start equals previous global_end for appends, and num_new_tokens <= kv_cache_size
2. Increase kv_cache_size to hold the largest chunk
3. Fix chunk-start bookkeeping in the caller (reset/rewind logic must keep indices consistent)

Example fix

# before
cache.update_and_get_attention_kv(k, v, current_chunk_start=rewind_pos + 1, ...)
# after
cache.update_and_get_attention_kv(k, v, current_chunk_start=rewind_pos, ...)

assert current_chunk_start == cache.global_end or current_chunk_start == cache.cur_start, 'non-sequential chunk'
assert num_new_tokens <= cache.kv_cache_size

try:\n    cache.update_and_get_attention_kv(k, v, current_chunk_start=s)\nexcept RuntimeError as e:\n    if 'Invalid' in str(e) and 'write range' in str(e):\n        cache.reset(); cache.update_and_get_attention_kv(k, v, current_chunk_start=s)

When it happens

Trigger: Calling update_and_get_attention_kv where current_chunk_start is far from the previous global_end, or num_new_tokens exceeds kv_cache_size, or a rewind/rewrite produces a window length different from num_new_tokens.

Common situations: Non-sequential chunk starts after a rollback/replay in diffusion or realtime generation; kv_cache_size configured smaller than a single chunk; off-by-one in current_chunk_start tracking across denoise steps.

Related errors

• kv-canary: SWA slot {slot} is outside full_to_swa_index_mapp
• {debug_name}: non-sequential write current_start={current_ch
• v_cache must be provided
• k_cache can only be None when only_qv=True
• rope_pool_fused expects pool tensors to be 3-D

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/63872d2966717096. Report an issue: GitHub.