sgl-project/sglang · error · NotImplementedError

{type(self).__name__} does not implement ring KV-chunk atten

Error message

{type(self).__name__} does not implement ring KV-chunk attention

What it means

AttentionBackend.forward_ring_kv_chunk is the abstract hook for one-rotated-KV-chunk attention used by ring-attention KV merging; it must return (output [Tq,H,D], lse [H,Tq]). Backends without ring support raise NotImplementedError explicitly.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/attention/backends/attention_backend.py:223

        max_seqlen: int,
        cu_seqlens_host: tuple[int, ...] | None = None,
    ) -> torch.Tensor:
        raise NotImplementedError(
            f"{type(self).__name__} does not implement packed varlen attention"
        )

    def forward_ring_kv_chunk(
        self,
        query: torch.Tensor,
        key: torch.Tensor,
        value: torch.Tensor,
    ) -> tuple[torch.Tensor, torch.Tensor]:
        """Attend local queries to one rotated KV chunk for ring merging.

        Inputs use packed ``[T, H, D]`` layout. The returned attention output
        has the query shape and softmax LSE uses ``[H, Tq]`` layout.
        """
        raise NotImplementedError(
            f"{type(self).__name__} does not implement ring KV-chunk attention"
        )


def wrap_attention_impl_forward(attn_impl: AttentionImpl) -> AttentionImpl:
    return wrap_method_with_debug_kernel_once(
        attn_impl,
        "forward",
        op_name=f"diffusion.attn_impl.{attn_impl.__class__.__name__}.forward",
    )

View on GitHub (pinned to 0132848349)

Solutions

1. Use a backend that implements ring KV-chunk attention (the flash_attn path with return_softmax_lse=True) for layers in ring merging
2. Disable ring attention / context-parallel KV merging for this backend in server args
3. Implement forward_ring_kv_chunk returning (output, lse[H, Tq]) if you own the backend

Example fix

# before
out, lse = backend.forward_ring_kv_chunk(q, k_chunk, v_chunk)  # NotImplementedError
# after
if type(backend).forward_ring_kv_chunk is AttentionBackend.forward_ring_kv_chunk:
    raise SystemExit(f"{type(backend).__name__} cannot run ring attention; use flash_attn backend")
out, lse = backend.forward_ring_kv_chunk(q, k_chunk, v_chunk)

def supports_ring(backend) -> bool:
    return type(backend).forward_ring_kv_chunk is not AttentionBackend.forward_ring_kv_chunk

def implements_ring_kv_chunk(b: AttentionBackend) -> bool:
    return type(b).forward_ring_kv_chunk is not AttentionBackend.forward_ring_kv_chunk

try:
    out, lse = backend.forward_ring_kv_chunk(q, kc, vc)
except NotImplementedError:
    disable_ring_attention_for(backend)

When it happens

Trigger: Running _ring_attention_varlen with a backend that does not override forward_ring_kv_chunk, e.g. sage_attn3, sliding_tile, or a custom backend.

Common situations: Enabling ring/context-parallel attention on a configuration whose backend only supports local attention; an SGLang upgrade making ring merge the default path for a backend that never implemented it.

Related errors

• Ring Attention requires one of the ring-capable backends ({'
• AITer backend does not have a metadata builder.
• AITER Sage backend does not have a metadata builder.
• NPU packed attention does not support a sequence that is emp
• {type(self).__name__} does not implement packed varlen atten

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/a947857bdc4c9c23. Report an issue: GitHub.