2noise/ChatTTS · critical · ValueError

The model's max seq len ({self.model_config.max_model_len})

Error message

The model's max seq len ({self.model_config.max_model_len}) is larger than the maximum number of tokens that can be stored in KV cache ({max_seq_len}). Try increasing `gpu_memory_utilization` or decreasing `max_model_len` when initializing the engine.

What it means

Even though some KV cache blocks exist, their total capacity (block_size * num_gpu_blocks) is smaller than max_model_len, so a single maximum-length sequence could never fit. The engine raises instead of deadlocking at runtime when a sequence exceeds cache capacity. Note the derived max_model_len (from config.json) counts here too, not just a user-passed value.

Source

Thrown at ChatTTS/model/velocity/llm_engine.py:290

        # Since we use a shared centralized controller, we take the minimum
        # number of blocks across all workers to make sure all the memory
        # operators can be applied to all workers.
        num_gpu_blocks = min(b[0] for b in num_blocks)
        num_cpu_blocks = min(b[1] for b in num_blocks)
        # FIXME(woosuk): Change to debug log.
        logger.info(
            f"# GPU blocks: {num_gpu_blocks}, " f"# CPU blocks: {num_cpu_blocks}"
        )

        if num_gpu_blocks <= 0:
            raise ValueError(
                "No available memory for the cache blocks. "
                "Try increasing `gpu_memory_utilization` when "
                "initializing the engine."
            )
        max_seq_len = self.cache_config.block_size * num_gpu_blocks
        if self.model_config.max_model_len > max_seq_len:
            raise ValueError(
                f"The model's max seq len ({self.model_config.max_model_len}) "
                "is larger than the maximum number of tokens that can be "
                f"stored in KV cache ({max_seq_len}). Try increasing "
                "`gpu_memory_utilization` or decreasing `max_model_len` when "
                "initializing the engine."
            )

        self.cache_config.num_gpu_blocks = num_gpu_blocks
        self.cache_config.num_cpu_blocks = num_cpu_blocks

        # Initialize the cache.
        self._run_workers("init_cache_engine", cache_config=self.cache_config)
        # Warm up the model. This includes capturing the model into CUDA graph
        # if enforce_eager is False.
        self._run_workers("warm_up_model")

    @classmethod
    def from_engine_args(

View on GitHub (pinned to 77b89ee281)

Solutions

1. Increase gpu_memory_utilization to reserve more memory for the KV cache.
2. Decrease max_model_len to fit within block_size * num_gpu_blocks (cap context to what you actually need).
3. Reduce memory pressure from weights: use a quantized checkpoint, or offload/reduce parallel per-GPU size.

Example fix

# before
engine = LLM(model=path, max_model_len=32768, gpu_memory_utilization=0.5)

# after
engine = LLM(model=path, max_model_len=8192, gpu_memory_utilization=0.9)

def feasible_max_model_len(block_size, num_gpu_blocks):
    return block_size * num_gpu_blocks
# after one successful init you can read engine.cache_config.num_gpu_blocks,
# then cap max_model_len accordingly for restarts

try:
    engine = LLM(model=path, max_model_len=want, gpu_memory_utilization=u)
except ValueError as e:
    if 'larger than the maximum number of tokens' in str(e):
        engine = LLM(model=path, max_model_len=want // 2, gpu_memory_utilization=min(0.95, u + 0.1))
    else:
        raise

When it happens

Trigger: Large max_model_len (e.g. 32k) on a GPU where profiling yields few blocks; small gpu_memory_utilization with a long-context model; long rope-scaled context on limited VRAM.

Common situations: Long-context fine-tunes on consumer GPUs; gpu_memory_utilization lowered to coexist with another job; 8k+ context models on 12-24GB cards.

Related errors

• User-specified max_model_len ({max_model_len}) is greater th

AI-assisted analysis of 2noise/ChatTTS@77b89ee281 (2026-08-26). Data as JSON: /api/errors/e4f0c0f49470db50. Report an issue: GitHub.