sgl-project/sglang · error · ValueError

Invalid deepep_mode: {self.deepep_mode}

Error message

Invalid deepep_mode: {self.deepep_mode}

What it means

DeepEPTokenDispatcher._get_impl selects the normal or low-latency sub-dispatcher based on the per-batch resolved DeepEPMode (auto mode resolves extend-vs-decode per batch). Any resolved value other than NORMAL or LOW_LATENCY — notably an unresolved AUTO — raises this ValueError. Called from dispatch_a/b and combine_a/b, so it fires at the first token step.

Source

Thrown at python/sglang/srt/layers/moe/token_dispatcher/deepep.py:1007

            topk_weights=topk_weights,
        )
        self._combine_intermediate_state = inner_state

    def combine_b(self):
        self._update_stage(_Stage.AFTER_COMBINE_A, _Stage.INITIAL)
        inner_state = self._combine_intermediate_state
        del self._combine_intermediate_state
        return self._get_impl().combine_b(*inner_state)

    def _get_impl(self) -> _DeepEPDispatcherImplBase:
        is_extend_in_batch = get_is_extend_in_batch()
        resolved_deepep_mode = self.deepep_mode.resolve(is_extend_in_batch)
        if resolved_deepep_mode == DeepEPMode.NORMAL:
            return self._normal_dispatcher
        elif resolved_deepep_mode == DeepEPMode.LOW_LATENCY:
            return self._low_latency_dispatcher
        else:
            raise ValueError(f"Invalid deepep_mode: {self.deepep_mode}")

    def _update_stage(self, old_stage, new_stage):
        assert self._stage == old_stage
        self._stage = new_stage

    def set_quant_config(self, quant_config: dict):
        super().set_quant_config(quant_config)
        if self.deepep_mode.enable_low_latency():
            self._low_latency_dispatcher.set_quant_config(quant_config)
        if self.deepep_mode.enable_normal():
            self._normal_dispatcher.set_quant_config(quant_config)

    def set_overlap_args(
        self, combine_overlap_args: CombineOverlapArgs, meta_overlap_args: dict
    ):
        super().set_overlap_args(combine_overlap_args, meta_overlap_args)
        if self.deepep_mode.enable_low_latency():
            self._low_latency_dispatcher.set_overlap_args(

View on GitHub (pinned to 0132848349)

Solutions

1. Upgrade SGLang so all components share one DeepEPMode definition
2. Pass an explicit non-auto mode: --deepep-mode normal or --deepep-mode low_latency instead of auto, bypassing the ambiguous resolve
3. If you added a DeepEPMode member, extend _get_impl to map it to a sub-dispatcher

Example fix

# before
--deepep-mode auto   # resolve() yields AUTO -> ValueError in _get_impl

# after
--deepep-mode low_latency  # or normal; explicit mode

from sglang.srt.layers.moe.token_dispatcher.deepep import DeepEPMode
if deepep_mode == DeepEPMode.AUTO:
    deepep_mode = DeepEPMode.AUTO_LOW_LATENCY  # or pass explicit --deepep-mode low_latency

When it happens

Trigger: self.deepep_mode.resolve(is_extend_in_batch) returning AUTO (custom/patched resolve, or a DeepEPMode constructed oddly), or a new enum member added without updating this if-chain; fires on the first dispatch call after construction/CUDA-graph capture.

Common situations: Version skew between sglang packages (python vs sgl-kernel vs a fork) where DeepEPMode gained new members; user subclasses overriding resolve(); pickled state from an older version.

Related errors

• unsupported mode
• norm_type must be one of "layer" and "rms"
• num_token_non_padded must be a single-element tensor, got sh
• num_token_non_padded must be an integer tensor, got {num_tok
• num_token_non_padded and x must be on the same device

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/04d948dbbb82c29a. Report an issue: GitHub.