sgl-project/sglang · error · RuntimeError

combine() called before dispatch()

Error message

combine() called before dispatch()

What it means

The Ascend TP token dispatcher caches the routing metadata from dispatch() and requires it in combine(); calling combine() when _dispatch_output is None means the state machine was violated. This is a programming/lifecycle error in the caller (or a scheduler bug), not a config problem.

Source

Thrown at python/sglang/srt/layers/moe/token_dispatcher/ascend_tp.py:130

            topk_ids,
            self.num_experts,
            top_k,
        )

        self._dispatch_output = AscendTPDispatchOutput(
            hidden_states=permuted_hidden_states,
            hidden_states_scale=hidden_states_scale,
            topk_weights=topk_weights,
            topk_ids=topk_ids,
            expanded_row_idx=expanded_row_idx,
            expert_tokens=expert_tokens,
            group_list_type=self.group_list_type,
        )
        return self._dispatch_output

    def combine(self, combine_input: AscendTPCombineInput) -> torch.Tensor:
        if self._dispatch_output is None:
            raise RuntimeError("combine() called before dispatch()")

        dispatch_out = self._dispatch_output

        # The finalizer (possibly wrapped with TP all‑gather) does all the work.
        final_hidden_states = self.finalize._finalize_routing(
            combine_input.hidden_states,
            topk_weights=dispatch_out.topk_weights,
            expanded_row_idx=dispatch_out.expanded_row_idx,
            topk_ids=dispatch_out.topk_ids,
        )

        self._dispatch_output = None
        return final_hidden_states

View on GitHub (pinned to 0132848349)

Solutions

1. Ensure the call order dispatch() -> (expert compute) -> combine() on the same dispatcher instance
2. Wrap dispatch in try/except and abort the step (do not proceed to combine) when dispatch fails
3. If writing a custom integration, mirror the pattern in sglang's token_dispatcher callers

Example fix

# before
disp = AscendTPDispatcher(...)
out = disp.combine(combine_input)  # RuntimeError

# after
dispatch_out = disp.dispatch(hidden_states, topk_output)
...  # expert compute
out = disp.combine(combine_input)

assert disp._dispatch_output is not None, "dispatch() must succeed before combine()"

try:
    dispatch_out = disp.dispatch(hidden_states, topk_output)
except Exception:
    abort_step()  # never fall through to combine()
else:
    out = disp.combine(combine_input)

When it happens

Trigger: Constructing AscendTPDispatcher and invoking combine(AscendTPCombineInput) without a prior successful dispatch(hidden_states, topk_output); also if dispatch raised midway and the error was swallowed, leaving state unset.

Common situations: Custom inference loops or scripted runtimes that skip/reorder dispatch; recovery code continuing after a swallowed dispatch exception; bugs in new dispatcher integrations.

Related errors

• Unsupported ascend_dispatcher_output_dtype: {self.ascend_dis
• {name} is required for NPU packed attention
• {name} must be a 1D int32 or int64 tensor
• {name} and its host copy must have the same length
• {name} must start with 0 and contain at least one sequence

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/e03ca327425ad005. Report an issue: GitHub.