sgl-project/sglang · error · RuntimeError

Dots SWA latent decode requires page_size=64, got

Error message

Dots SWA latent decode requires page_size=64, got {backend.page_size}.

What it means

The Dots hybrid SWA latent-decode path (forward_swa_mla_absorbed) assumes paged KV with page_size=64; its kernels index pages at that granularity. At runtime, if the selected backend's page_size differs, the forward raises RuntimeError before corrupting the KV cache.

Solutions

1. Launch with --page-size 64
2. Remove any page-size override so a compatible default is chosen, if the default is 64
3. Avoid routing Dots SWA latent decode through a backend with a different page size

Example fix

# before
--page-size 16  # Dots hybrid model
# after
--page-size 64

backend = self.selected_backend(forward_batch)
assert backend.page_size == 64, "launch Dots hybrid models with --page-size 64"

def dots_swa_compatible(page_size: int) -> bool:
    return page_size == 64

When it happens

Trigger: Calling forward_swa_mla_absorbed on a Dots hybrid model where backend.page_size != 64 — typically because the server was launched with --page-size 1/16/32 or the backend default resolved to another page size.

Common situations: Tuning page size for other models then serving a Dots hybrid checkpoint; mixing FlashInfer-style page-size overrides with the Dots SWA MLA kernels.

Related errors

• Cross attention is not supported in the hpc_ops attention…
• Currently only attention backends are supported for…
• cutedsl_mla backend can only be used with MLA models.
• Decode attention backend for Kimi-K3 DCP must be…
• Dual chunk attention is enabled, but attention backend is…

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/516b4344f4f43b8b. Report an issue: GitHub.

Appendix: source

Thrown at python/sglang/srt/layers/attention/dots_hybrid_backend.py:363

            max_seqlen_k=metadata.max_seq_len_k,
            softmax_scale=layer.scaling,
            causal=True,
            window_size=(layer.sliding_window_size, 0),
            ver=self._active_backend.fa_impl_ver,
        )
        if pad_v_to_qk:
            output = output[..., : layer.v_head_dim]
        return output.reshape(-1, layer.tp_q_head_num * layer.v_head_dim)

    def forward_swa_mla_absorbed(self, q, layer, forward_batch):
        """Run decode directly against the page64 latent SWA cache."""
        from sglang.srt.layers.attention.swa_mla_fallback.forward import (
            forward_dense_kvlora_swa_torch_fallback,
        )

        backend = self.selected_backend(forward_batch)
        if backend.page_size != 64:
            raise RuntimeError(
                "Dots SWA latent decode requires page_size=64, "
                f"got {backend.page_size}."
            )

        self.maybe_rebuild_metadata_after_dp_padding(forward_batch)
        metadata = backend.forward_metadata
        block_table = metadata.swa_page_table
        if block_table is None:
            raise RuntimeError("Dots SWA latent decode requires an SWA page table.")
        bs = forward_batch.batch_size
        block_table = _normalize_page_table_rows(block_table, bs)
        cache_seqlens = _normalize_cache_seqlens_rows(
            metadata.cache_seqlens_int32,
            forward_batch.seq_lens,
            bs,
        )
        reshape_q = q.view(bs, -1, layer.tp_q_head_num, layer.head_dim)
        k_cache = self.token_to_kv_pool.get_key_buffer(layer.layer_id)

View on GitHub (pinned to 0132848349)