{"record":{"id":"f29368ca3fc19717","repo":"sgl-project/sglang","slug":"type-self-name-does-not-implement-packed-va","errorCode":null,"errorMessage":"{type(self).__name__} does not implement packed varlen attention","messagePattern":"(.+?) does not implement packed varlen attention","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/layers/attention/backends/attention_backend.py","lineNumber":208,"sourceCode":"        self,\n        query: torch.Tensor,\n        key: torch.Tensor,\n        value: torch.Tensor,\n        attn_metadata: T,\n    ) -> torch.Tensor:\n        raise NotImplementedError\n\n    def forward_varlen(\n        self,\n        query: torch.Tensor,\n        key: torch.Tensor,\n        value: torch.Tensor,\n        *,\n        cu_seqlens: torch.Tensor,\n        max_seqlen: int,\n        cu_seqlens_host: tuple[int, ...] | None = None,\n    ) -> torch.Tensor:\n        raise NotImplementedError(\n            f\"{type(self).__name__} does not implement packed varlen attention\"\n        )\n\n    def forward_ring_kv_chunk(\n        self,\n        query: torch.Tensor,\n        key: torch.Tensor,\n        value: torch.Tensor,\n    ) -> tuple[torch.Tensor, torch.Tensor]:\n        \"\"\"Attend local queries to one rotated KV chunk for ring merging.\n\n        Inputs use packed ``[T, H, D]`` layout. The returned attention output\n        has the query shape and softmax LSE uses ``[H, Tq]`` layout.\n        \"\"\"\n        raise NotImplementedError(\n            f\"{type(self).__name__} does not implement ring KV-chunk attention\"\n        )\n","sourceCodeStart":190,"sourceCodeEnd":226,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/layers/attention/backends/attention_backend.py#L190-L226","documentation":"AttentionBackend.forward_varlen is an abstract hook for packed variable-length (THD) attention. Backends that only support batched inputs deliberately raise NotImplementedError so the gap surfaces early instead of silently producing wrong results.","triggerScenarios":"Calling forward_varlen(query, key, value, cu_seqlens=..., max_seqlen=...) on a backend subclass that did not override it, e.g. SlidingTileAttentionBackend or a custom backend implementing only forward().","commonSituations":"Switching attention_backend to one without varlen support while the workload uses packed sequences; new custom backends copied from a batched-only template; multimodal/diffusion models routed to a backend with only forward().","solutions":["Switch to a backend that implements packed varlen attention (flash_attn, ascend_fa, etc.) via server args / attention_backend_config","If you own the backend, implement forward_varlen handling cu_seqlens/max_seqlen packed [T, H, D] tensors","Route layers needing varlen to a supported backend and keep the limited backend only for forward() layers"],"exampleFix":"# before\nout = backend.forward_varlen(q, k, v, cu_seqlens=cu, max_seqlen=m)  # NotImplementedError\n# after\nif type(backend).forward_varlen is AttentionBackend.forward_varlen:\n    out = run_padded_attention(backend, q, k, v, cu)\nelse:\n    out = backend.forward_varlen(q, k, v, cu_seqlens=cu, max_seqlen=m)","handlingStrategy":"type-guard","validationCode":"from sglang.multimodal_gen.runtime.layers.attention.backends.attention_backend import AttentionBackend\n\ndef has_varlen(backend) -> bool:\n    return type(backend).forward_varlen is not AttentionBackend.forward_varlen","typeGuard":"def implements_varlen(b: AttentionBackend) -> bool:\n    return type(b).forward_varlen is not AttentionBackend.forward_varlen","tryCatchPattern":"try:\n    out = backend.forward_varlen(q, k, v, cu_seqlens=cu, max_seqlen=m)\nexcept NotImplementedError:\n    out = run_padded_attention(backend, q, k, v, cu)","preventionTips":["Check backend capability before routing packed batches","Keep an explicit matrix of backend vs supported forward modes"],"tags":["attention-backend","varlen","not-implemented","abstract-method"],"backgroundTag":"not-implemented-abstract-method","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}