2noise/ChatTTS · error · ValueError

User-specified max_model_len ({max_model_len}) is greater th

Error message

User-specified max_model_len ({max_model_len}) is greater than the derived max_model_len ({max_len_key}={derived_max_model_len} in model's config.json). This may lead to incorrect model outputs or CUDA errors. Make sure the value is correct and within the model context size.

What it means

The engine derives a maximum context length from the model's config.json (max_position_embeddings, or original_max_position_embeddings * scaling_factor when rope scaling is set). If you pass max_model_len larger than that derived value, the config raises rather than silently running beyond the trained context. This protects against garbage outputs or CUDA errors from out-of-range positions.

Source

Thrown at ChatTTS/model/velocity/configs.py:539

            "The model's config.json does not contain any of the following "
            "keys to determine the original maximum length of the model: "
            f"{possible_keys}. Assuming the model's maximum length is "
            f"{default_max_len}."
        )
        derived_max_model_len = default_max_len

    rope_scaling = getattr(hf_config, "rope_scaling", None)
    if rope_scaling is not None:
        assert "factor" in rope_scaling
        scaling_factor = rope_scaling["factor"]
        if rope_scaling["type"] == "yarn":
            derived_max_model_len = rope_scaling["original_max_position_embeddings"]
        derived_max_model_len *= scaling_factor

    if max_model_len is None:
        max_model_len = derived_max_model_len
    elif max_model_len > derived_max_model_len:
        raise ValueError(
            f"User-specified max_model_len ({max_model_len}) is greater than "
            f"the derived max_model_len ({max_len_key}={derived_max_model_len}"
            " in model's config.json). This may lead to incorrect model "
            "outputs or CUDA errors. Make sure the value is correct and "
            "within the model context size."
        )
    return int(max_model_len)


@dataclass
class EngineArgs:
    """Arguments for vLLM engine."""

    model: str
    tokenizer: Optional[str] = None
    tokenizer_mode: str = "auto"
    trust_remote_code: bool = False
    download_dir: Optional[str] = None

View on GitHub (pinned to 77b89ee281)

Solutions

Lower max_model_len to <= the derived value from config.json (max_position_embeddings, or original_max_position_embeddings * scaling_factor with rope scaling).
If you truly need longer context, load a checkpoint that was actually extended (e.g. a rope-scaled fine-tune with higher original_max_position_embeddings/scaling_factor).
If the config.json value is wrong (model actually supports more), edit config.json deliberately - not recommended unless you own the checkpoint.

Example fix

# before
engine = LLM(model=path, max_model_len=8192)  # config has max_position_embeddings=4096

# after
engine = LLM(model=path, max_model_len=4092)  # <= derived length

Defensive patterns

Strategy: validation

Validate before calling

import json

def safe_max_model_len(model_path, requested):
    cfg = json.load(open(f'{model_path}/config.json'))
    derived = cfg.get('max_position_embeddings')
    if cfg.get('rope_scaling'):
        rs = cfg['rope_scaling']
        derived = rs.get('original_max_position_embeddings', derived) * rs.get('factor', 1.0)
    return min(requested, derived) if requested else derived

Try / catch

try:
    engine = LLM(model=path, max_model_len=want)
except ValueError as e:
    if 'greater than the derived max_model_len' in str(e):
        engine = LLM(model=path)  # let engine derive it
    else:
        raise

Prevention

Read max_position_embeddings (and rope_scaling) from config.json before choosing max_model_len.
Treat long-context numbers from READMEs as checkpoint-specific, not universal.

When it happens

Trigger: LLM(..., max_model_len=8192) on a model whose config.json has max_position_embeddings=4096; or with rope_scaling whose original_max_position_embeddings * scaling_factor is smaller than the requested max_model_len.

Common situations: Copying a max_model_len from a long-context fine-tune (e.g. 32k) while loading the base checkpoint; mistaking rope scaling factor for an automatic context extension; config.json drift after re-exporting a merged model.

Related errors

AI-assisted analysis of 2noise/ChatTTS@77b89ee281 (2026-08-26). Data as JSON: /api/errors/6870dbb8097d9f4f. Report an issue: GitHub.