2noise/ChatTTS · error · ValueError

The quantization method {model_config.quantization} is not s

Error message

The quantization method {model_config.quantization} is not supported for the current GPU. Minimum capability: {quant_config.get_min_capability()}. Current capability: {capability}.

What it means

Quantized models (e.g. AWQ, GPTQ) ship kernels that require a minimum CUDA compute capability. get_model reads torch.cuda.get_device_capability(), packs it as major*10+minor, and compares against the quantization config's minimum; below it, init aborts because dequant/matmul kernels would crash or be absent. This is the first of two gates - act-dtype checks follow.

Source

Thrown at ChatTTS/model/velocity/model_loader.py:37

    torch.set_default_dtype(dtype)
    yield
    torch.set_default_dtype(old_dtype)


def get_model(model_config: ModelConfig) -> nn.Module:
    # Get the (maybe quantized) linear method.
    linear_method = None
    if model_config.quantization is not None:
        quant_config = get_quant_config(
            model_config.quantization,
            model_config.model,
            model_config.hf_config,
            model_config.download_dir,
        )
        capability = torch.cuda.get_device_capability()
        capability = capability[0] * 10 + capability[1]
        if capability < quant_config.get_min_capability():
            raise ValueError(
                f"The quantization method {model_config.quantization} is not "
                "supported for the current GPU. "
                f"Minimum capability: {quant_config.get_min_capability()}. "
                f"Current capability: {capability}."
            )
        supported_dtypes = quant_config.get_supported_act_dtypes()
        if model_config.dtype not in supported_dtypes:
            raise ValueError(
                f"{model_config.dtype} is not supported for quantization "
                f"method {model_config.quantization}. Supported dtypes: "
                f"{supported_dtypes}"
            )
        linear_method = quant_config.get_linear_method()

    with _set_default_torch_dtype(model_config.dtype):
        # Create a model instance.
        # The weights will be initialized as empty tensors.
        with torch.device("cuda"):

View on GitHub (pinned to 77b89ee281)

Solutions

Use the unquantized (fp16) checkpoint - pass no quantization argument - on this GPU.
Switch to a GPU with compute capability >= the stated minimum (commonly 7.5 Turing or 8.0 Ampere).
Pick a quantization method whose min capability matches your hardware (check quant_config.get_min_capability() for each method).

Example fix

# before
engine = LLM(model=awq_path, quantization='awq')  # on GTX 1080 (cap 6.1)

# after
engine = LLM(model=fp16_path)  # unquantized weights work on old GPUs

Defensive patterns

Strategy: validation

Validate before calling

import torch

def gpu_capability():
    c = torch.cuda.get_device_capability()
    return c[0] * 10 + c[1]

MIN_CAP = {'awq': 75, 'gptq': 75}  # verify against your quant config

def quant_ok(method):
    return method is None or gpu_capability() >= MIN_CAP.get(method, 0)

Try / catch

try:
    engine = LLM(model=quant_path, quantization='awq')
except ValueError as e:
    if 'not supported for the current GPU' in str(e):
        engine = LLM(model=fp16_path)  # unquantized fallback
    else:
        raise

Prevention

Print torch.cuda.get_device_capability() before choosing quantization.
Keep an unquantized checkpoint around for older GPUs.

When it happens

Trigger: Loading an AWQ/GPTQ quantized model on an old GPU (e.g. compute capability 6.1 Pascal, or 7.0 for methods requiring 7.5+) whose packed capability is below quant_config.get_min_capability().

Common situations: Running AWQ/GPTQ checkpoints on GTX 10-series or Tesla P100/V100; mixing T4-era quantized checkpoints with older local hardware; ChatTTS velocity engine configured with quantization='awq' on a pre-Turing card.

Related errors

AI-assisted analysis of 2noise/ChatTTS@77b89ee281 (2026-08-26). Data as JSON: /api/errors/e9acd76337a5f397. Report an issue: GitHub.