{"record":{"id":"e4f0c0f49470db50","repo":"2noise/ChatTTS","slug":"the-model-s-max-seq-len-self-model-config-max-mo","errorCode":null,"errorMessage":"The model's max seq len ({self.model_config.max_model_len}) is larger than the maximum number of tokens that can be stored in KV cache ({max_seq_len}). Try increasing `gpu_memory_utilization` or decreasing `max_model_len` when initializing the engine.","messagePattern":"The model's max seq len \\((.+?)\\) is larger than the maximum number of tokens that can be stored in KV cache \\((.+?)\\)\\. Try increasing `gpu_memory_utilization` or decreasing `max_model_len` when initializing the engine\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"critical","filePath":"ChatTTS/model/velocity/llm_engine.py","lineNumber":290,"sourceCode":"        # Since we use a shared centralized controller, we take the minimum\n        # number of blocks across all workers to make sure all the memory\n        # operators can be applied to all workers.\n        num_gpu_blocks = min(b[0] for b in num_blocks)\n        num_cpu_blocks = min(b[1] for b in num_blocks)\n        # FIXME(woosuk): Change to debug log.\n        logger.info(\n            f\"# GPU blocks: {num_gpu_blocks}, \" f\"# CPU blocks: {num_cpu_blocks}\"\n        )\n\n        if num_gpu_blocks <= 0:\n            raise ValueError(\n                \"No available memory for the cache blocks. \"\n                \"Try increasing `gpu_memory_utilization` when \"\n                \"initializing the engine.\"\n            )\n        max_seq_len = self.cache_config.block_size * num_gpu_blocks\n        if self.model_config.max_model_len > max_seq_len:\n            raise ValueError(\n                f\"The model's max seq len ({self.model_config.max_model_len}) \"\n                \"is larger than the maximum number of tokens that can be \"\n                f\"stored in KV cache ({max_seq_len}). Try increasing \"\n                \"`gpu_memory_utilization` or decreasing `max_model_len` when \"\n                \"initializing the engine.\"\n            )\n\n        self.cache_config.num_gpu_blocks = num_gpu_blocks\n        self.cache_config.num_cpu_blocks = num_cpu_blocks\n\n        # Initialize the cache.\n        self._run_workers(\"init_cache_engine\", cache_config=self.cache_config)\n        # Warm up the model. This includes capturing the model into CUDA graph\n        # if enforce_eager is False.\n        self._run_workers(\"warm_up_model\")\n\n    @classmethod\n    def from_engine_args(","sourceCodeStart":272,"sourceCodeEnd":308,"githubUrl":"https://github.com/2noise/ChatTTS/blob/77b89ee281cd479f5b1a787ada330dc975ca1f2a/ChatTTS/model/velocity/llm_engine.py#L272-L308","documentation":"Even though some KV cache blocks exist, their total capacity (block_size * num_gpu_blocks) is smaller than max_model_len, so a single maximum-length sequence could never fit. The engine raises instead of deadlocking at runtime when a sequence exceeds cache capacity. Note the derived max_model_len (from config.json) counts here too, not just a user-passed value.","triggerScenarios":"Large max_model_len (e.g. 32k) on a GPU where profiling yields few blocks; small gpu_memory_utilization with a long-context model; long rope-scaled context on limited VRAM.","commonSituations":"Long-context fine-tunes on consumer GPUs; gpu_memory_utilization lowered to coexist with another job; 8k+ context models on 12-24GB cards.","solutions":["Increase gpu_memory_utilization to reserve more memory for the KV cache.","Decrease max_model_len to fit within block_size * num_gpu_blocks (cap context to what you actually need).","Reduce memory pressure from weights: use a quantized checkpoint, or offload/reduce parallel per-GPU size."],"exampleFix":"# before\nengine = LLM(model=path, max_model_len=32768, gpu_memory_utilization=0.5)\n\n# after\nengine = LLM(model=path, max_model_len=8192, gpu_memory_utilization=0.9)","handlingStrategy":"validation","validationCode":"def feasible_max_model_len(block_size, num_gpu_blocks):\n    return block_size * num_gpu_blocks\n# after one successful init you can read engine.cache_config.num_gpu_blocks,\n# then cap max_model_len accordingly for restarts","typeGuard":null,"tryCatchPattern":"try:\n    engine = LLM(model=path, max_model_len=want, gpu_memory_utilization=u)\nexcept ValueError as e:\n    if 'larger than the maximum number of tokens' in str(e):\n        engine = LLM(model=path, max_model_len=want // 2, gpu_memory_utilization=min(0.95, u + 0.1))\n    else:\n        raise","preventionTips":["Cap max_model_len to the shortest context your workload actually needs.","Recompute cache feasibility after changing gpu_memory_utilization or checkpoint size."],"tags":["gpu-memory","kv-cache","max-model-len","context-length"],"backgroundTag":"gpu-out-of-memory","analyzedSha":"77b89ee281cd479f5b1a787ada330dc975ca1f2a","analyzedAt":"2026-08-26T17:48:24.233Z","schemaVersion":2},"datasetVersion":"2026-08-26T21:11:00.512Z"}