sgl-project/sglang · error · ValueError

Missing adaptive runtime state for steps={speculative_num_st

Error message

Missing adaptive runtime state for steps={speculative_num_steps}

What it means

AdaptiveRuntimeState._activate looks up a runtime state by speculative step count and finds none registered. The adaptive spec-decoding runtime must have states pre-created for every candidate step value before activation.

Source

Thrown at python/sglang/srt/speculative/adaptive_runtime_state.py:131

    def activate_step_by_batch(self, batch_size: int) -> None:
        target = self.params.get_steps_for_batch(batch_size)
        if target != self.worker.speculative_num_steps:
            self._activate(target)

    def on_verify_complete(
        self, num_correct_drafts_per_req: list[int], batch_size: int
    ) -> None:
        """Feed verify results; switch runtime state if EMA warrants it."""
        new_step = self.params.on_verify_complete(
            num_correct_drafts_per_req, batch_size
        )
        if new_step is not None:
            self._activate(new_step)

    def _activate(self, speculative_num_steps: int) -> None:
        state = self._states.get(speculative_num_steps)
        if state is None:
            raise ValueError(
                f"Missing adaptive runtime state for steps={speculative_num_steps}"
            )
        self.worker.apply_runtime_state(state)

View on GitHub (pinned to 0132848349)

Solutions

1. Ensure speculative_adaptive_config's candidate_steps covers every step value the runtime can select
2. Call init_states (full re-initialization) after changing candidate steps so all states are pre-built
3. Check for stale configs on the worker vs scheduler side

Example fix

# before
{"1": {"candidate_steps": [1, 3]}}
# after
{"1": {"candidate_steps": [1, 3, 5, 7]}}  # include 5 if runtime may select it

allowed = set(candidate_steps_from_config)
assert requested_steps in allowed, f'steps={requested_steps} not in {sorted(allowed)}'

catch ValueError from _activate and trigger init_states rebuild once, then retry

When it happens

Trigger: Calling activate/apply paths with a speculative_num_steps that was never registered in _states — e.g. a config lists candidate_steps [1,3,7] but the scheduler later requests steps=5.

Common situations: Mismatch between speculative_adaptive_config candidate_steps and the actual steps chosen at runtime; partially initialized worker; adding a new step count without rebuilding states (init_states not called after config change).

Related errors

• Kimi-K3 DCP + DSPARK currently requires SGLANG_RAGGED_VERIFY
• --disaggregation-decode-enable-radix-cache is incompatible w
• Gemma4AssistantForCausalLM draft requires --speculative-algo
• --speculative-draft-window-size must be positive, got {}.
• DFLASH speculative decoding only supports CUDA and NPU devic

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/b85047f626c3a8b1. Report an issue: GitHub.