{"record":{"id":"e9acd76337a5f397","repo":"2noise/ChatTTS","slug":"the-quantization-method-model-config-quantization","errorCode":null,"errorMessage":"The quantization method {model_config.quantization} is not supported for the current GPU. Minimum capability: {quant_config.get_min_capability()}. Current capability: {capability}.","messagePattern":"The quantization method (.+?) is not supported for the current GPU\\. Minimum capability: (.+?)\\. Current capability: (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"ChatTTS/model/velocity/model_loader.py","lineNumber":37,"sourceCode":"    torch.set_default_dtype(dtype)\n    yield\n    torch.set_default_dtype(old_dtype)\n\n\ndef get_model(model_config: ModelConfig) -> nn.Module:\n    # Get the (maybe quantized) linear method.\n    linear_method = None\n    if model_config.quantization is not None:\n        quant_config = get_quant_config(\n            model_config.quantization,\n            model_config.model,\n            model_config.hf_config,\n            model_config.download_dir,\n        )\n        capability = torch.cuda.get_device_capability()\n        capability = capability[0] * 10 + capability[1]\n        if capability < quant_config.get_min_capability():\n            raise ValueError(\n                f\"The quantization method {model_config.quantization} is not \"\n                \"supported for the current GPU. \"\n                f\"Minimum capability: {quant_config.get_min_capability()}. \"\n                f\"Current capability: {capability}.\"\n            )\n        supported_dtypes = quant_config.get_supported_act_dtypes()\n        if model_config.dtype not in supported_dtypes:\n            raise ValueError(\n                f\"{model_config.dtype} is not supported for quantization \"\n                f\"method {model_config.quantization}. Supported dtypes: \"\n                f\"{supported_dtypes}\"\n            )\n        linear_method = quant_config.get_linear_method()\n\n    with _set_default_torch_dtype(model_config.dtype):\n        # Create a model instance.\n        # The weights will be initialized as empty tensors.\n        with torch.device(\"cuda\"):","sourceCodeStart":19,"sourceCodeEnd":55,"githubUrl":"https://github.com/2noise/ChatTTS/blob/77b89ee281cd479f5b1a787ada330dc975ca1f2a/ChatTTS/model/velocity/model_loader.py#L19-L55","documentation":"Quantized models (e.g. AWQ, GPTQ) ship kernels that require a minimum CUDA compute capability. get_model reads torch.cuda.get_device_capability(), packs it as major*10+minor, and compares against the quantization config's minimum; below it, init aborts because dequant/matmul kernels would crash or be absent. This is the first of two gates - act-dtype checks follow.","triggerScenarios":"Loading an AWQ/GPTQ quantized model on an old GPU (e.g. compute capability 6.1 Pascal, or 7.0 for methods requiring 7.5+) whose packed capability is below quant_config.get_min_capability().","commonSituations":"Running AWQ/GPTQ checkpoints on GTX 10-series or Tesla P100/V100; mixing T4-era quantized checkpoints with older local hardware; ChatTTS velocity engine configured with quantization='awq' on a pre-Turing card.","solutions":["Use the unquantized (fp16) checkpoint - pass no quantization argument - on this GPU.","Switch to a GPU with compute capability >= the stated minimum (commonly 7.5 Turing or 8.0 Ampere).","Pick a quantization method whose min capability matches your hardware (check quant_config.get_min_capability() for each method)."],"exampleFix":"# before\nengine = LLM(model=awq_path, quantization='awq')  # on GTX 1080 (cap 6.1)\n\n# after\nengine = LLM(model=fp16_path)  # unquantized weights work on old GPUs","handlingStrategy":"validation","validationCode":"import torch\n\ndef gpu_capability():\n    c = torch.cuda.get_device_capability()\n    return c[0] * 10 + c[1]\n\nMIN_CAP = {'awq': 75, 'gptq': 75}  # verify against your quant config\n\ndef quant_ok(method):\n    return method is None or gpu_capability() >= MIN_CAP.get(method, 0)","typeGuard":null,"tryCatchPattern":"try:\n    engine = LLM(model=quant_path, quantization='awq')\nexcept ValueError as e:\n    if 'not supported for the current GPU' in str(e):\n        engine = LLM(model=fp16_path)  # unquantized fallback\n    else:\n        raise","preventionTips":["Print torch.cuda.get_device_capability() before choosing quantization.","Keep an unquantized checkpoint around for older GPUs."],"tags":["quantization","awq","gptq","cuda-capability","gpu"],"backgroundTag":"gpu-compute-capability-unsupported","analyzedSha":"77b89ee281cd479f5b1a787ada330dc975ca1f2a","analyzedAt":"2026-08-26T17:48:24.233Z","schemaVersion":2},"datasetVersion":"2026-08-26T21:11:00.512Z"}