sgl-project/sglang · error · ValueError
--enable-two-batch-overlap is not supported with DSA…
Error message
--enable-two-batch-overlap is not supported with DSA index-topk sharing (index_topk_freq > 1 or an index_topk_pattern containing shared layers): the TBO op path does not propagate topk indices across layers, so shared layers would run sparse attention without indices.
What it means
Two-batch overlap (TBO) is incompatible with DSA index-topk sharing (index_topk_freq > 1 or an index_topk_pattern containing shared 'S' layers): the TBO op path does not propagate topk indices across layers, so shared layers would run sparse attention without indices and produce wrong results. ServerArgs fails fast instead.
Solutions
- Remove --enable-two-batch-overlap for DSA models with index-topk sharing
- Or use a checkpoint/config with index_topk_freq == 1 and no shared layers in index_topk_pattern
- Track SGLang releases for TBO support of shared topk indices
Example fix
# before --enable-two-batch-overlap # DSA model with index_topk_freq=2 # after # (flag removed)
Defensive patterns
Strategy: validation
Validate before calling
freq = getattr(hf_config, "index_topk_freq", 1) or 1
pat = getattr(hf_config, "index_topk_pattern", None)
if args.get("enable_two_batch_overlap") and (freq > 1 or (pat and "S" in pat)):
args["enable_two_batch_overlap"] = False Prevention
- Inspect the DSA checkpoint's index_topk settings before enabling TBO
- Treat throughput flags like TBO as per-checkpoint, not per-cluster, settings
When it happens
Trigger: Serving a DSA model whose hf_config has index_topk_freq > 1 or index_topk_pattern with shared layers while passing --enable-two-batch-overlap.
Common situations: Enabling TBO for throughput on a DSA checkpoint that shares topk indices between layers; flag carried over from a non-sharing DSA config after a checkpoint upgrade.
Related errors
- DSA indexer only supports CUDA, HIP, and NPU
- DSA indexer weights_proj LoRA is incompatible with…
- --enable-dsa-cache-layer-split is only supported for DSA…
- Required: indexer, forward_batch, x, q_lora, positions
- allow_neg_eigval=True requires 2*sigmoid(beta), which is…
AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28).
Data as JSON: /api/errors/d50d436c46ec5961.
Report an issue: GitHub.
Appendix: source
Thrown at python/sglang/srt/server_args.py:5879
# When threshold is not manually set, set it to the index topk of model
from sglang.srt.configs.model_config import get_dsa_index_topk
envs.SGLANG_DSA_PREFILL_DENSE_ATTN_KV_LEN_THRESHOLD.set(
get_dsa_index_topk(hf_config)
)
logger.warning(
f"Set dense attention kv len threshold to model index_topk={envs.SGLANG_DSA_PREFILL_DENSE_ATTN_KV_LEN_THRESHOLD.get()} for DeepSeek with DSA."
)
# The "dsa" attention fill moved to the override registry
# (arg_groups/overrides.py: _deepseek_family_overrides).
index_topk_freq = getattr(hf_config, "index_topk_freq", 1) or 1
index_topk_pattern = getattr(hf_config, "index_topk_pattern", None)
if cfg.enable_two_batch_overlap and (
index_topk_freq > 1
or (index_topk_pattern is not None and "S" in index_topk_pattern)
):
raise ValueError(
"--enable-two-batch-overlap is not supported with DSA "
"index-topk sharing (index_topk_freq > 1 or an "
"index_topk_pattern containing shared layers): the TBO op "
"path does not propagate topk indices across layers, so "
"shared layers would run sparse attention without indices."
)
if not is_npu() and not is_xpu(): # CUDA or ROCm GPU
if cfg.enable_prefill_cp:
# The DSA CP field declarations moved to the override
# registry (arg_groups/overrides.py:
# _deepseek_family_overrides).
self._declare(
"_handle_model_specific_adjustments",
cuda_graph_config=with_phase(
cfg.cuda_graph_config,
Phase.PREFILL,
backend=Backend.DISABLED,View on GitHub (pinned to 0132848349)