sgl-project/sglang · error · NotImplementedError

QVGPackedCausalKVCache does not support pinned-sink (longliv

Error message

QVGPackedCausalKVCache does not support pinned-sink (longlive2); packed KV quant is scoped to the LingBot realtime path.

What it means

QVGPackedCausalKVCache deliberately does not implement pinned-sink behavior (used by the 'longlive2' long-context mode); calling pin_current_chunk raises NotImplementedError, since packed KV quant is only intended for the LingBot realtime path.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/kvcache/qvg_packed_cache.py:157

        if self.global_end_index_int is not None:
            self.global_end_index_int = 0
            self.local_end_index_int = 0
        self.global_end_index.zero_()
        self.local_end_index.zero_()

    def can_direct_current_attention(self, num_new_tokens: int) -> bool:
        return (
            self.sink_tokens == 0
            and self.cache_size == num_new_tokens
            and self.attention_window_size == num_new_tokens
        )

    @property
    def num_cache_heads(self) -> int:
        return self.num_heads

    def pin_current_chunk(self, current_num_tokens: int) -> None:
        raise NotImplementedError(
            "QVGPackedCausalKVCache does not support pinned-sink (longlive2); "
            "packed KV quant is scoped to the LingBot realtime path."
        )

    def resident_nbytes(self) -> int:
        total = sum(s.nbytes() for s in self._segments)
        if self._cur is not None:
            total += self._cur.nbytes()
        return total

    # -------------------------------------------------------------- helpers
    def _new_bf16_segment(self, g0: int, g1: int, is_sink: bool) -> _Segment:
        n = g1 - g0
        k = torch.zeros(
            self.batch_size,
            n,
            self.num_heads,
            self.head_dim,

View on GitHub (pinned to 0132848349)

Solutions

1. Disable KV quantization (use the bf16 packed cache) when running longlive2/pinned-sink
2. Or disable pinned-sink mode if you need packed KV quant (LingBot realtime path)
3. Add an explicit config guard so the two features cannot be combined at startup

Example fix

# before
cache = QVGPackedCausalKVCache(..., quant=True)
cache.pin_current_chunk(n)  # NotImplementedError
# after: use non-quantized cache for longlive2
cache = PackedCausalKVCache(...)  # supports pinned sink

if enable_pinned_sink and enable_kv_quant:
    raise ValueError('pinned-sink (longlive2) is incompatible with packed KV quant')

def supports_pin(cache) -> bool:\n    return callable(getattr(cache, 'pin_current_chunk', None)) and 'QVG' not in type(cache).__name__

try:\n    cache.pin_current_chunk(n)\nexcept NotImplementedError:\n    logger.warning('pinned sink unsupported by %s', type(cache).__name__)

When it happens

Trigger: Calling pin_current_chunk on a QVGPackedCausalKVCache instance — i.e. enabling the longlive2/pinned-sink feature together with the quantized packed KV cache.

Common situations: Config that enables KV quantization while also enabling pinned-sink long-context mode; reusing a longlive2 cache abstraction across cache implementations that don't support it.

Related errors

• {self.__class__.__name__}.apply should not be called.
• kv_scales supplied but unified_kv is {unified_kv.dtype}, exp
• Quant-VideoGen KV-cache quantization requires its optional r
• {debug_name}: non-sequential write current_start={current_ch
• bitsandbytes 4-bit TP only supports column-parallel output s

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/4b9b09fc9cf2519b. Report an issue: GitHub.