sgl-project/sglang · error · ValueError

Dual chunk attention is enabled, but attention backend is se

Error message

Dual chunk attention is enabled, but attention backend is set to {}. Please set it to 'dual_chunk_flash_attn'.

What it means

When dual chunk attention is enabled (e.g. via --enable-dual-chunk-attention for models like Gemma with interleaved attention), the attention backend must be dual_chunk_flash_attn. If an explicit different backend is already set, SGLang raises this conflict instead of silently overriding it.

Source

Thrown at python/sglang/srt/arg_groups/overrides.py:2642

        if view.page_size not in supported_page_sizes:
            logger.warning(
                f"{msg} only supports page_sizes of {supported_page_sizes}, changing page_size from {view.page_size} to 128."
            )
            return {"page_size": 128}
    return {}


@register_post_process
def _attention_backend_dual_chunk(view: Any) -> dict:
    if (
        getattr(view.get_model_config().hf_config, "dual_chunk_attention_config", None)
        is not None
    ):
        if view.attention_backend is None:
            logger.info("Dual chunk attention is turned on by default.")
            return {"attention_backend": "dual_chunk_flash_attn"}
        elif view.attention_backend != "dual_chunk_flash_attn":
            raise ValueError(
                "Dual chunk attention is enabled, but attention backend is set to "
                f"{view.attention_backend}. Please set it to 'dual_chunk_flash_attn'."
            )
    return {}


@register_post_process
def _page_size_default(view: Any) -> dict:
    if view.page_size is not None:
        return {}

    # SHUFFLE 5D vectorized KV layout (aiter backend + pa_decode_gluon)
    # is tuned for and prefers page_size=64 — making it the default
    # when the layout flag is set avoids users having to pass
    # --page-size 64 explicitly. The env var is only consumed by the
    # ROCm AITER backend, so the auto-bump is gated on HIP; on other
    # platforms the SHUFFLE 5D pool has no consumer kernels and the
    # env var is silently ignored (see MHATokenToKVPool).

View on GitHub (pinned to 0132848349)

Solutions

1. Remove the explicit --attention-backend flag so the default dual_chunk_flash_attn is applied
2. Or explicitly set --attention-backend dual_chunk_flash_attn
3. Disable dual chunk attention if the model does not need it

Example fix

# before
--enable-dual-chunk-attention --attention-backend fa3
# after
--enable-dual-chunk-attention --attention-backend dual_chunk_flash_attn

if args.enable_dual_chunk_attention and args.attention_backend not in (None, "dual_chunk_flash_attn"):
    args.attention_backend = "dual_chunk_flash_attn"

When it happens

Trigger: Enabling dual chunk attention while --attention-backend is explicitly set to something other than dual_chunk_flash_attn (fa3, flashinfer, trtllm_mla, ...).

Common situations: A launch script that always sets an attention backend being reused with a dual-chunk-attention model; setting a backend globally for a fleet where one model needs dual chunk attention.

Related errors

• {selection_error}{component_suffix}
• No compatible attention backend is available{component_suffi
• Subclass {self.__class__.__name__} must define _supported_at
• HiSparse supports DSA {label} backend(s) {sorted(allowed_bac
• Decode attention backend for Kimi-K3 DCP must be 'cutedsl_ml

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/18d8249853b8d72d. Report an issue: GitHub.