{"record":{"id":"29e2e30b57fcd9f3","repo":"vllm-project/vllm","slug":"msg-to-allow-overriding-this-maximum-set-the-en","errorCode":null,"errorMessage":"{msg} To allow overriding this maximum, set the env var VLLM_ALLOW_LONG_MAX_MODEL_LEN=1. {warning}","messagePattern":"(.+?) To allow overriding this maximum, set the env var VLLM_ALLOW_LONG_MAX_MODEL_LEN=1\\. (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/config/model.py","lineNumber":2461,"sourceCode":"        if model_max_length is None or max_model_len > model_max_length:\n            msg = (\n                f\"User-specified max_model_len ({max_model_len}) is greater \"\n                f\"than the derived max_model_len ({max_len_key}=\"\n                f\"{derived_max_model_len} or model_max_length=\"\n                f\"{model_max_length} in model's config.json).\"\n            )\n            warning = (\n                \"VLLM_ALLOW_LONG_MAX_MODEL_LEN must be used with extreme \"\n                \"caution. If the model uses relative position encoding (RoPE), \"\n                \"positions exceeding derived_max_model_len lead to nan. If the \"\n                \"model uses absolute position encoding, positions exceeding \"\n                \"derived_max_model_len will cause a CUDA array out-of-bounds \"\n                \"error.\"\n            )\n            if envs.VLLM_ALLOW_LONG_MAX_MODEL_LEN:\n                logger.warning_once(\"%s %s\", msg, warning)\n            else:\n                raise ValueError(\n                    f\"{msg} To allow overriding this maximum, set \"\n                    f\"the env var VLLM_ALLOW_LONG_MAX_MODEL_LEN=1. {warning}\"\n                )\n    return int(max_model_len)\n","sourceCodeStart":2443,"sourceCodeEnd":2466,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/config/model.py#L2443-L2466","documentation":"Final guard in _get_and_verify_max_len: when max_model_len exceeds the derived maximum (from config, tokenizer, sliding window, or KV cache hints) and VLLM_ALLOW_LONG_MAX_MODEL_LEN is unset, vLLM raises, including a warning about position-encoding overflow (NaN for RoPE, OOB for absolute encoding). With the env var set it only warns.","triggerScenarios":"Passing --max-model-len greater than the model's derived_max_model_len while envs.VLLM_ALLOW_LONG_MAX_MODEL_LEN is falsy — e.g. requesting 128k on a model whose config derives 32k.","commonSituations":"Trying to serve longer contexts than a checkpoint was trained for; misreading a tokenizer's max length as the model's; sliding-window models where derived length is the window, not the full sequence.","solutions":["Lower --max-model-len to at most the derived maximum reported in the message.","If you accept NaN/OOB risk, set env VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 before starting vLLM (it then only warns).","Use a long-context checkpoint (or a rope-scaled fine-tune) that actually supports the target length."],"exampleFix":"# before\nvllm serve my-model --max-model-len 131072\n# after\nvllm serve my-model --max-model-len 32768\n# or, accepting risk:\nVLLM_ALLOW_LONG_MAX_MODEL_LEN=1 vllm serve my-model --max-model-len 131072","handlingStrategy":"fallback","validationCode":"import os\ndef clamp_max_model_len(requested: int | None, derived_max: int) -> int:\n    if requested is None or requested <= derived_max:\n        return requested or derived_max\n    if os.environ.get('VLLM_ALLOW_LONG_MAX_MODEL_LEN') == '1':\n        return requested  # operator accepted NaN/OOB risk\n    return derived_max","typeGuard":null,"tryCatchPattern":"except ValueError as e:\n    if 'VLLM_ALLOW_LONG_MAX_MODEL_LEN' in str(e):\n        retry with max_model_len capped to the derived maximum from the error message","preventionTips":["Derive max_model_len from the model config (min of derived max and your serving budget) instead of hardcoding.","Only set VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 for models known to tolerate extrapolation (e.g. rope-scaled).","Remember sliding-window models derive max len from the window, not the training length."],"tags":["max-model-len","context-length","config","startup"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}