{"record":{"id":"7db53798dc3d9b28","repo":"sgl-project/sglang","slug":"debug-name-current-chunk-rewrite-size-changed","errorCode":null,"errorMessage":"{debug_name}: current-chunk rewrite size changed","messagePattern":"(.+?): current-chunk rewrite size changed","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/layers/kvcache/qvg_packed_cache.py","lineNumber":295,"sourceCode":"        cache_head_start: int | None = None,\n        recent_window_tokens: int | None = None,\n        debug_name: str = \"QVG packed KV cache\",\n    ) -> CausalAttentionKVView:\n        num_new = key.shape[1]\n        num_input_heads = key.shape[2]\n        head_slice = None\n        if num_input_heads != self.num_heads:\n            if cache_head_start is None:\n                raise ValueError(\n                    f\"{debug_name}: cache_head_start required for head slice\"\n                )\n            head_slice = slice(cache_head_start, cache_head_start + num_input_heads)\n        cend = current_chunk_start + num_new\n\n        if self._cur is not None and current_chunk_start == self._cur.g0:\n            # rewrite current chunk in place (denoise step)\n            if cend != self._cur.g1:\n                raise NotImplementedError(\n                    f\"{debug_name}: current-chunk rewrite size changed\"\n                )\n            self._write(self._cur, key, value, head_slice)\n        elif current_chunk_start == self._global_end:\n            # advance: finalize current chunk, start a new one\n            if self._cur is not None:\n                self._segments.append(self._cur)\n            is_sink = current_chunk_start < self.sink_tokens\n            if self._chunk_tokens == 0:\n                self._chunk_tokens = num_new\n            self._cur = self._new_bf16_segment(current_chunk_start, cend, is_sink)\n            self._write(self._cur, key, value, head_slice)\n            self._global_end = cend\n            self._pack_and_evict()\n        else:\n            raise NotImplementedError(\n                f\"{debug_name}: non-sequential write current_start=\"\n                f\"{current_chunk_start} global_end={self._global_end} \"","sourceCodeStart":277,"sourceCodeEnd":313,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/layers/kvcache/qvg_packed_cache.py#L277-L313","documentation":"During a denoise rewrite of the current chunk (current_chunk_start == self._cur.g0), the packed cache requires the rewrite to cover exactly the same token span as before (cend == g1). A different length means the chunk shape changed mid-flight, which the segment layout cannot accommodate.","triggerScenarios":"Calling update_and_get_attention_kv twice with the same current_chunk_start but key/value tensors with different sequence lengths (different num_new across denoise steps).","commonSituations":"Diffusion denoise loops where chunk length varies between steps (e.g. condition tokens appended on some steps); bug in chunk length computation producing off-by-N lengths.","solutions":["Keep the rewrite length identical across denoise steps for the same chunk start","Recompute/verify num_new = key.shape[1] is stable per chunk","If variable-length rewrites are needed, start a new chunk instead of rewriting (advance global_end)"],"exampleFix":"# before: step2 writes fewer tokens at same start\ncache.update_and_get_attention_kv(k2[:, :, :n2], v2, current_chunk_start=s)\n# after: same length as step1\ncache.update_and_get_attention_kv(k2, v2, current_chunk_start=s)  # n2 == n1","handlingStrategy":"validation","validationCode":"if current_chunk_start == cache.cur_start:\n    assert num_new == cache.cur_len, 'rewrite must keep chunk length'","typeGuard":null,"tryCatchPattern":"try:\\n    cache.update_and_get_attention_kv(k, v, current_chunk_start=s)\\nexcept NotImplementedError:\\n    cache.start_new_chunk(s)  # or reset","preventionTips":["Fix chunk length per chunk start across denoise steps","Track expected chunk length alongside positions"],"tags":["kv-cache","diffusion","rewrite","shape-mismatch"],"backgroundTag":"kv-cache-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}