{"record":{"id":"4b9b09fc9cf2519b","repo":"sgl-project/sglang","slug":"qvgpackedcausalkvcache-does-not-support-pinned-sin","errorCode":null,"errorMessage":"QVGPackedCausalKVCache does not support pinned-sink (longlive2); packed KV quant is scoped to the LingBot realtime path.","messagePattern":"QVGPackedCausalKVCache does not support pinned-sink \\(longlive2\\); packed KV quant is scoped to the LingBot realtime path\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/layers/kvcache/qvg_packed_cache.py","lineNumber":157,"sourceCode":"        if self.global_end_index_int is not None:\n            self.global_end_index_int = 0\n            self.local_end_index_int = 0\n        self.global_end_index.zero_()\n        self.local_end_index.zero_()\n\n    def can_direct_current_attention(self, num_new_tokens: int) -> bool:\n        return (\n            self.sink_tokens == 0\n            and self.cache_size == num_new_tokens\n            and self.attention_window_size == num_new_tokens\n        )\n\n    @property\n    def num_cache_heads(self) -> int:\n        return self.num_heads\n\n    def pin_current_chunk(self, current_num_tokens: int) -> None:\n        raise NotImplementedError(\n            \"QVGPackedCausalKVCache does not support pinned-sink (longlive2); \"\n            \"packed KV quant is scoped to the LingBot realtime path.\"\n        )\n\n    def resident_nbytes(self) -> int:\n        total = sum(s.nbytes() for s in self._segments)\n        if self._cur is not None:\n            total += self._cur.nbytes()\n        return total\n\n    # -------------------------------------------------------------- helpers\n    def _new_bf16_segment(self, g0: int, g1: int, is_sink: bool) -> _Segment:\n        n = g1 - g0\n        k = torch.zeros(\n            self.batch_size,\n            n,\n            self.num_heads,\n            self.head_dim,","sourceCodeStart":139,"sourceCodeEnd":175,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/layers/kvcache/qvg_packed_cache.py#L139-L175","documentation":"QVGPackedCausalKVCache deliberately does not implement pinned-sink behavior (used by the 'longlive2' long-context mode); calling pin_current_chunk raises NotImplementedError, since packed KV quant is only intended for the LingBot realtime path.","triggerScenarios":"Calling pin_current_chunk on a QVGPackedCausalKVCache instance — i.e. enabling the longlive2/pinned-sink feature together with the quantized packed KV cache.","commonSituations":"Config that enables KV quantization while also enabling pinned-sink long-context mode; reusing a longlive2 cache abstraction across cache implementations that don't support it.","solutions":["Disable KV quantization (use the bf16 packed cache) when running longlive2/pinned-sink","Or disable pinned-sink mode if you need packed KV quant (LingBot realtime path)","Add an explicit config guard so the two features cannot be combined at startup"],"exampleFix":"# before\ncache = QVGPackedCausalKVCache(..., quant=True)\ncache.pin_current_chunk(n)  # NotImplementedError\n# after: use non-quantized cache for longlive2\ncache = PackedCausalKVCache(...)  # supports pinned sink","handlingStrategy":"validation","validationCode":"if enable_pinned_sink and enable_kv_quant:\n    raise ValueError('pinned-sink (longlive2) is incompatible with packed KV quant')","typeGuard":"def supports_pin(cache) -> bool:\\n    return callable(getattr(cache, 'pin_current_chunk', None)) and 'QVG' not in type(cache).__name__","tryCatchPattern":"try:\\n    cache.pin_current_chunk(n)\\nexcept NotImplementedError:\\n    logger.warning('pinned sink unsupported by %s', type(cache).__name__)","preventionTips":["Declare feature matrix per cache implementation","Reject incompatible flag combos at config parse time"],"tags":["kv-cache","not-implemented","feature-incompatibility","quantization"],"backgroundTag":"unsupported-feature-combination","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}