sgl-project/sglang · error · ValueError

--enable-two-batch-overlap is not supported with DSA index-t

Error message

--enable-two-batch-overlap is not supported with DSA index-topk sharing (index_topk_freq > 1 or an index_topk_pattern containing shared layers): the TBO op path does not propagate topk indices across layers, so shared layers would run sparse attention without indices.

What it means

Two-batch overlap (TBO) is incompatible with DSA index-topk sharing (index_topk_freq > 1 or an index_topk_pattern containing shared 'S' layers): the TBO op path does not propagate topk indices across layers, so shared layers would run sparse attention without indices and produce wrong results. ServerArgs fails fast instead.

Source

Thrown at python/sglang/srt/server_args.py:5879

                    # When threshold is not manually set, set it to the index topk of model
                    from sglang.srt.configs.model_config import get_dsa_index_topk

                    envs.SGLANG_DSA_PREFILL_DENSE_ATTN_KV_LEN_THRESHOLD.set(
                        get_dsa_index_topk(hf_config)
                    )
                    logger.warning(
                        f"Set dense attention kv len threshold to model index_topk={envs.SGLANG_DSA_PREFILL_DENSE_ATTN_KV_LEN_THRESHOLD.get()} for DeepSeek with DSA."
                    )
                # The "dsa" attention fill moved to the override registry
                # (arg_groups/overrides.py: _deepseek_family_overrides).

                index_topk_freq = getattr(hf_config, "index_topk_freq", 1) or 1
                index_topk_pattern = getattr(hf_config, "index_topk_pattern", None)
                if cfg.enable_two_batch_overlap and (
                    index_topk_freq > 1
                    or (index_topk_pattern is not None and "S" in index_topk_pattern)
                ):
                    raise ValueError(
                        "--enable-two-batch-overlap is not supported with DSA "
                        "index-topk sharing (index_topk_freq > 1 or an "
                        "index_topk_pattern containing shared layers): the TBO op "
                        "path does not propagate topk indices across layers, so "
                        "shared layers would run sparse attention without indices."
                    )

                if not is_npu() and not is_xpu():  # CUDA or ROCm GPU
                    if cfg.enable_prefill_cp:
                        # The DSA CP field declarations moved to the override
                        # registry (arg_groups/overrides.py:
                        # _deepseek_family_overrides).
                        self._declare(
                            "_handle_model_specific_adjustments",
                            cuda_graph_config=with_phase(
                                cfg.cuda_graph_config,
                                Phase.PREFILL,
                                backend=Backend.DISABLED,

View on GitHub (pinned to 0132848349)

Solutions

1. Remove --enable-two-batch-overlap for DSA models with index-topk sharing
2. Or use a checkpoint/config with index_topk_freq == 1 and no shared layers in index_topk_pattern
3. Track SGLang releases for TBO support of shared topk indices

Example fix

# before
--enable-two-batch-overlap  # DSA model with index_topk_freq=2
# after
# (flag removed)

freq = getattr(hf_config, "index_topk_freq", 1) or 1
pat = getattr(hf_config, "index_topk_pattern", None)
if args.get("enable_two_batch_overlap") and (freq > 1 or (pat and "S" in pat)):
    args["enable_two_batch_overlap"] = False

When it happens

Trigger: Serving a DSA model whose hf_config has index_topk_freq > 1 or index_topk_pattern with shared layers while passing --enable-two-batch-overlap.

Common situations: Enabling TBO for throughput on a DSA checkpoint that shares topk indices between layers; flag carried over from a non-sharing DSA config after a checkpoint upgrade.

Related errors

• DSA indexer weights_proj LoRA is incompatible with piecewise
• DSA indexer only supports CUDA, HIP, and NPU
• Required: indexer, forward_batch, x, q_lora, positions
• --enable-dsa-cache-layer-split is only supported for DSA (De
• sparse_attn_v4_paged_decode expects fp16/bf16 q, got {q.dtyp

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/d50d436c46ec5961. Report an issue: GitHub.