{"record":{"id":"6870dbb8097d9f4f","repo":"2noise/ChatTTS","slug":"user-specified-max-model-len-max-model-len-is","errorCode":null,"errorMessage":"User-specified max_model_len ({max_model_len}) is greater than the derived max_model_len ({max_len_key}={derived_max_model_len} in model's config.json). This may lead to incorrect model outputs or CUDA errors. Make sure the value is correct and within the model context size.","messagePattern":"User-specified max_model_len \\((.+?)\\) is greater than the derived max_model_len \\((.+?)=(.+?) in model's config\\.json\\)\\. This may lead to incorrect model outputs or CUDA errors\\. Make sure the value is correct and within the model context size\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"ChatTTS/model/velocity/configs.py","lineNumber":539,"sourceCode":"            \"The model's config.json does not contain any of the following \"\n            \"keys to determine the original maximum length of the model: \"\n            f\"{possible_keys}. Assuming the model's maximum length is \"\n            f\"{default_max_len}.\"\n        )\n        derived_max_model_len = default_max_len\n\n    rope_scaling = getattr(hf_config, \"rope_scaling\", None)\n    if rope_scaling is not None:\n        assert \"factor\" in rope_scaling\n        scaling_factor = rope_scaling[\"factor\"]\n        if rope_scaling[\"type\"] == \"yarn\":\n            derived_max_model_len = rope_scaling[\"original_max_position_embeddings\"]\n        derived_max_model_len *= scaling_factor\n\n    if max_model_len is None:\n        max_model_len = derived_max_model_len\n    elif max_model_len > derived_max_model_len:\n        raise ValueError(\n            f\"User-specified max_model_len ({max_model_len}) is greater than \"\n            f\"the derived max_model_len ({max_len_key}={derived_max_model_len}\"\n            \" in model's config.json). This may lead to incorrect model \"\n            \"outputs or CUDA errors. Make sure the value is correct and \"\n            \"within the model context size.\"\n        )\n    return int(max_model_len)\n\n\n@dataclass\nclass EngineArgs:\n    \"\"\"Arguments for vLLM engine.\"\"\"\n\n    model: str\n    tokenizer: Optional[str] = None\n    tokenizer_mode: str = \"auto\"\n    trust_remote_code: bool = False\n    download_dir: Optional[str] = None","sourceCodeStart":521,"sourceCodeEnd":557,"githubUrl":"https://github.com/2noise/ChatTTS/blob/77b89ee281cd479f5b1a787ada330dc975ca1f2a/ChatTTS/model/velocity/configs.py#L521-L557","documentation":"The engine derives a maximum context length from the model's config.json (max_position_embeddings, or original_max_position_embeddings * scaling_factor when rope scaling is set). If you pass max_model_len larger than that derived value, the config raises rather than silently running beyond the trained context. This protects against garbage outputs or CUDA errors from out-of-range positions.","triggerScenarios":"LLM(..., max_model_len=8192) on a model whose config.json has max_position_embeddings=4096; or with rope_scaling whose original_max_position_embeddings * scaling_factor is smaller than the requested max_model_len.","commonSituations":"Copying a max_model_len from a long-context fine-tune (e.g. 32k) while loading the base checkpoint; mistaking rope scaling factor for an automatic context extension; config.json drift after re-exporting a merged model.","solutions":["Lower max_model_len to <= the derived value from config.json (max_position_embeddings, or original_max_position_embeddings * scaling_factor with rope scaling).","If you truly need longer context, load a checkpoint that was actually extended (e.g. a rope-scaled fine-tune with higher original_max_position_embeddings/scaling_factor).","If the config.json value is wrong (model actually supports more), edit config.json deliberately - not recommended unless you own the checkpoint."],"exampleFix":"# before\nengine = LLM(model=path, max_model_len=8192)  # config has max_position_embeddings=4096\n\n# after\nengine = LLM(model=path, max_model_len=4092)  # <= derived length","handlingStrategy":"validation","validationCode":"import json\n\ndef safe_max_model_len(model_path, requested):\n    cfg = json.load(open(f'{model_path}/config.json'))\n    derived = cfg.get('max_position_embeddings')\n    if cfg.get('rope_scaling'):\n        rs = cfg['rope_scaling']\n        derived = rs.get('original_max_position_embeddings', derived) * rs.get('factor', 1.0)\n    return min(requested, derived) if requested else derived","typeGuard":null,"tryCatchPattern":"try:\n    engine = LLM(model=path, max_model_len=want)\nexcept ValueError as e:\n    if 'greater than the derived max_model_len' in str(e):\n        engine = LLM(model=path)  # let engine derive it\n    else:\n        raise","preventionTips":["Read max_position_embeddings (and rope_scaling) from config.json before choosing max_model_len.","Treat long-context numbers from READMEs as checkpoint-specific, not universal."],"tags":["max-model-len","context-length","rope-scaling","configuration"],"backgroundTag":"context-length-exceeds-model-limit","analyzedSha":"77b89ee281cd479f5b1a787ada330dc975ca1f2a","analyzedAt":"2026-08-26T17:48:24.233Z","schemaVersion":2},"datasetVersion":"2026-08-26T21:11:00.512Z"}