sgl-project/sglang · error · NotImplementedError

{debug_name}: current-chunk rewrite size changed

Error message

{debug_name}: current-chunk rewrite size changed

What it means

During a denoise rewrite of the current chunk (current_chunk_start == self._cur.g0), the packed cache requires the rewrite to cover exactly the same token span as before (cend == g1). A different length means the chunk shape changed mid-flight, which the segment layout cannot accommodate.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/kvcache/qvg_packed_cache.py:295

        cache_head_start: int | None = None,
        recent_window_tokens: int | None = None,
        debug_name: str = "QVG packed KV cache",
    ) -> CausalAttentionKVView:
        num_new = key.shape[1]
        num_input_heads = key.shape[2]
        head_slice = None
        if num_input_heads != self.num_heads:
            if cache_head_start is None:
                raise ValueError(
                    f"{debug_name}: cache_head_start required for head slice"
                )
            head_slice = slice(cache_head_start, cache_head_start + num_input_heads)
        cend = current_chunk_start + num_new

        if self._cur is not None and current_chunk_start == self._cur.g0:
            # rewrite current chunk in place (denoise step)
            if cend != self._cur.g1:
                raise NotImplementedError(
                    f"{debug_name}: current-chunk rewrite size changed"
                )
            self._write(self._cur, key, value, head_slice)
        elif current_chunk_start == self._global_end:
            # advance: finalize current chunk, start a new one
            if self._cur is not None:
                self._segments.append(self._cur)
            is_sink = current_chunk_start < self.sink_tokens
            if self._chunk_tokens == 0:
                self._chunk_tokens = num_new
            self._cur = self._new_bf16_segment(current_chunk_start, cend, is_sink)
            self._write(self._cur, key, value, head_slice)
            self._global_end = cend
            self._pack_and_evict()
        else:
            raise NotImplementedError(
                f"{debug_name}: non-sequential write current_start="
                f"{current_chunk_start} global_end={self._global_end} "

View on GitHub (pinned to 0132848349)

Solutions

1. Keep the rewrite length identical across denoise steps for the same chunk start
2. Recompute/verify num_new = key.shape[1] is stable per chunk
3. If variable-length rewrites are needed, start a new chunk instead of rewriting (advance global_end)

Example fix

# before: step2 writes fewer tokens at same start
cache.update_and_get_attention_kv(k2[:, :, :n2], v2, current_chunk_start=s)
# after: same length as step1
cache.update_and_get_attention_kv(k2, v2, current_chunk_start=s)  # n2 == n1

if current_chunk_start == cache.cur_start:
    assert num_new == cache.cur_len, 'rewrite must keep chunk length'

try:\n    cache.update_and_get_attention_kv(k, v, current_chunk_start=s)\nexcept NotImplementedError:\n    cache.start_new_chunk(s)  # or reset

When it happens

Trigger: Calling update_and_get_attention_kv twice with the same current_chunk_start but key/value tensors with different sequence lengths (different num_new across denoise steps).

Common situations: Diffusion denoise loops where chunk length varies between steps (e.g. condition tokens appended on some steps); bug in chunk length computation producing off-by-N lengths.

Related errors

• kv d_qk must match q d_qk={d_qk}, got {kv_d_qk}
• kv-canary: launch_canary_plan_kernels_torch_reference verify
• {debug_name} requires cache_head_start when cache heads ({nu
• Position map {height}x{width} is not divisible by {grid_reso
• Hunyuan3D Paint expects square latents and a matching view c

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/7db53798dc3d9b28. Report an issue: GitHub.