vllm-project/vllm · error · ValueError

max_num_batched_tokens ({self.max_num_batched_tokens}) must

Error message

max_num_batched_tokens ({self.max_num_batched_tokens}) must be greater than or equal to max_num_seqs ({self.max_num_seqs}).

What it means

verify_max_model_len also requires max_num_batched_tokens >= max_num_seqs: each running sequence must be able to receive at least one token per scheduler step within the batched-token budget. Otherwise some sequences would be starved indefinitely.

Source

Thrown at vllm/config/scheduler.py:264

        self.verify_max_model_len(max_model_len)

    def verify_max_model_len(self, max_model_len: int) -> Self:
        if (
            self.max_num_batched_tokens < max_model_len
            and not self.enable_chunked_prefill
        ):
            raise ValueError(
                f"max_num_batched_tokens ({self.max_num_batched_tokens}) is "
                f"smaller than max_model_len ({max_model_len}). "
                "This effectively limits the maximum sequence length to "
                "max_num_batched_tokens and makes vLLM reject longer "
                "sequences. Please increase max_num_batched_tokens or "
                "decrease max_model_len."
            )

        if self.max_num_batched_tokens < self.max_num_seqs:
            raise ValueError(
                f"max_num_batched_tokens ({self.max_num_batched_tokens}) must "
                "be greater than or equal to max_num_seqs "
                f"({self.max_num_seqs})."
            )

        if self.max_num_batched_tokens > self.max_num_seqs * max_model_len:
            logger.warning(
                "max_num_batched_tokens (%d) exceeds max_num_seqs "
                "* max_model_len (%d). This may lead to unexpected behavior.",
                self.max_num_batched_tokens,
                self.max_num_seqs * max_model_len,
            )

        if self.long_prefill_token_threshold > max_model_len:
            raise ValueError(
                "long_prefill_token_threshold "
                f"({self.long_prefill_token_threshold}) cannot be greater "
                f"than the max_model_len ({max_model_len})."

View on GitHub (pinned to c794754062)

Solutions

1. Raise --max-num-batched-tokens to >= max_num_seqs
2. Lower --max-num-seqs to <= max_num_batched_tokens
3. Scale both together (e.g. budget = seqs * tokens-per-step) in deployment templates

Example fix

# before
--max-num-batched-tokens 64 --max-num-seqs 256

# after
--max-num-batched-tokens 4096 --max-num-seqs 256

def seqs_fit_budget(mnbt: int, max_num_seqs: int) -> bool:
    return mnbt >= max_num_seqs

null

null

When it happens

Trigger: --max-num-batched-tokens 64 with --max-num-seqs 256; deriving one of the two from GPU memory formulas independently so they cross; setting max_num_seqs high for throughput on small-token budgets.

Common situations: Aggressive concurrency tuning; copying configs between hardware profiles; defaults that assume a larger token budget than the operator set.

Related errors

• max_num_batched_tokens ({self.max_num_batched_tokens}) is sm
• long_prefill_token_threshold ({self.long_prefill_token_thres
• Unknown dtype: {dtype!r}
• 'mm_shm_cache_max_object_size_mb' should only be set when 'm
• 'mm_encoder_fp8_scale_path' and 'mm_encoder_fp8_scale_save_p

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/3ca77c0599b73dfa. Report an issue: GitHub.