get_num_tokens_per_bs_for_target_verify is deprecated; use g

Error message

get_num_tokens_per_bs_for_target_verify is deprecated; use get_num_tokens_per_req_for_target_verify instead.

What it means

Speculative decoding code emits this DeprecationWarning when the old batch-size-oriented API get_num_tokens_per_bs_for_target_verify is called. It has been renamed to get_num_tokens_per_req_for_target_verify to reflect per-request semantics; the old name is a thin alias that warns and forwards. It appears in python/sglang/srt/speculative/spec_info.py on the base SpecInfo class.

Source

Thrown at python/sglang/srt/speculative/spec_info.py:244

            _handle_ngram(server_args)

    def get_num_tokens_per_req_for_target_verify(
        self, num_draft_tokens: int, is_draft_worker: bool
    ) -> int:
        # FIXME: Remove this after the forward mode refactor. Target verify is
        # essentially a fixed sequence length prefill/extend with full cuda
        # graph support. We can use it for target verify, or we can use it for
        # other cases which is not target verify but fixed length prefill.
        # Here, we expose this interface to allow the other use cases.
        if self.is_dspark() and is_draft_worker:
            return num_draft_tokens - 1
        return num_draft_tokens

    def get_num_tokens_per_bs_for_target_verify(
        self, num_draft_tokens: int, is_draft_worker: bool
    ) -> int:
        # Deprecated alias; remove together with the FIXME above.
        warnings.warn(
            "get_num_tokens_per_bs_for_target_verify is deprecated; use "
            "get_num_tokens_per_req_for_target_verify instead.",
            DeprecationWarning,
            stacklevel=2,
        )
        return self.get_num_tokens_per_req_for_target_verify(
            num_draft_tokens, is_draft_worker
        )

    def create_worker(
        self, server_args: ServerArgs
    ) -> Optional[Union[Type[BaseSpecWorker], Type[TpModelWorker], Type[NGRAMWorker]]]:
        from sglang.srt.arg_groups.overrides import resolving_view

        cfg = resolving_view(server_args)
        assert (
            not self.is_none()
        ), "Cannot create worker for NONE speculative algorithm."

View on GitHub (pinned to 0132848349)

Solutions

Replace every call to get_num_tokens_per_bs_for_target_verify(...) with get_num_tokens_per_req_for_target_verify(...) — identical signature and return
If third-party code must keep working, run with -W ignore::DeprecationWarning or PYTHONWARNINGS=ignore::DeprecationWarning as a stopgap
After migrating, verify no remaining references: grep -r 'get_num_tokens_per_bs_for_target_verify' your_code/

Example fix

# before
n = spec_info.get_num_tokens_per_bs_for_target_verify(num_draft_tokens, is_draft_worker)
# after
n = spec_info.get_num_tokens_per_req_for_target_verify(num_draft_tokens, is_draft_worker)

Defensive patterns

Strategy: validation

Validate before calling

import warnings
with warnings.catch_warnings():
    warnings.simplefilter("ignore", DeprecationWarning)
    n = spec.get_num_tokens_per_bs_for_target_verify(k, is_draft)

Type guard

hasattr(spec, "get_num_tokens_per_req_for_target_verify")

Prevention

Migrate call sites to the _per_req_ name at upgrade time
Run tests with -W error::DeprecationWarning once to enumerate all stale call sites

When it happens

Trigger: Any code path (typically older draft-worker/target-verify batch preparation, custom spec algorithms, or downstream forks) invoking spec_info.get_num_tokens_per_bs_for_target_verify(num_draft_tokens, is_draft_worker). Triggered during scheduler preparation of target-verify batches in speculative decoding.

Common situations: Upgrading SGLang to a version where the bs->req rename happened while custom speculative algorithms or monkey-patches still call the old name; plugins/tests written against the older API surface.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/01687c2f3f5e1c72. Report an issue: GitHub.