vllm-project/vllm · error · ValueError

{msg} To allow overriding this maximum, set the env var VLLM

Error message

{msg} To allow overriding this maximum, set the env var VLLM_ALLOW_LONG_MAX_MODEL_LEN=1. {warning}

What it means

Final guard in _get_and_verify_max_len: when max_model_len exceeds the derived maximum (from config, tokenizer, sliding window, or KV cache hints) and VLLM_ALLOW_LONG_MAX_MODEL_LEN is unset, vLLM raises, including a warning about position-encoding overflow (NaN for RoPE, OOB for absolute encoding). With the env var set it only warns.

Source

Thrown at vllm/config/model.py:2461

        if model_max_length is None or max_model_len > model_max_length:
            msg = (
                f"User-specified max_model_len ({max_model_len}) is greater "
                f"than the derived max_model_len ({max_len_key}="
                f"{derived_max_model_len} or model_max_length="
                f"{model_max_length} in model's config.json)."
            )
            warning = (
                "VLLM_ALLOW_LONG_MAX_MODEL_LEN must be used with extreme "
                "caution. If the model uses relative position encoding (RoPE), "
                "positions exceeding derived_max_model_len lead to nan. If the "
                "model uses absolute position encoding, positions exceeding "
                "derived_max_model_len will cause a CUDA array out-of-bounds "
                "error."
            )
            if envs.VLLM_ALLOW_LONG_MAX_MODEL_LEN:
                logger.warning_once("%s %s", msg, warning)
            else:
                raise ValueError(
                    f"{msg} To allow overriding this maximum, set "
                    f"the env var VLLM_ALLOW_LONG_MAX_MODEL_LEN=1. {warning}"
                )
    return int(max_model_len)

View on GitHub (pinned to c794754062)

Solutions

Lower --max-model-len to at most the derived maximum reported in the message.
If you accept NaN/OOB risk, set env VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 before starting vLLM (it then only warns).
Use a long-context checkpoint (or a rope-scaled fine-tune) that actually supports the target length.

Example fix

# before
vllm serve my-model --max-model-len 131072
# after
vllm serve my-model --max-model-len 32768
# or, accepting risk:
VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 vllm serve my-model --max-model-len 131072

Defensive patterns

Strategy: fallback

Validate before calling

import os
def clamp_max_model_len(requested: int | None, derived_max: int) -> int:
    if requested is None or requested <= derived_max:
        return requested or derived_max
    if os.environ.get('VLLM_ALLOW_LONG_MAX_MODEL_LEN') == '1':
        return requested  # operator accepted NaN/OOB risk
    return derived_max

Try / catch

except ValueError as e:
    if 'VLLM_ALLOW_LONG_MAX_MODEL_LEN' in str(e):
        retry with max_model_len capped to the derived maximum from the error message

Prevention

Derive max_model_len from the model config (min of derived max and your serving budget) instead of hardcoding.
Only set VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 for models known to tolerate extrapolation (e.g. rope-scaled).
Remember sliding-window models derive max len from the window, not the training length.

When it happens

Trigger: Passing --max-model-len greater than the model's derived_max_model_len while envs.VLLM_ALLOW_LONG_MAX_MODEL_LEN is falsy — e.g. requesting 128k on a model whose config derives 32k.

Common situations: Trying to serve longer contexts than a checkpoint was trained for; misreading a tokenizer's max length as the model's; sliding-window models where derived length is the window, not the full sequence.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/29e2e30b57fcd9f3. Report an issue: GitHub.