2noise/ChatTTS · error · ValueError

{model_config.dtype} is not supported for quantization metho

Error message

{model_config.dtype} is not supported for quantization method {model_config.quantization}. Supported dtypes: {supported_dtypes}

What it means

Raised during model loading when a quantization method (e.g. awq, gptq, squeezellm) is configured but the requested activation dtype is not one the quant config supports. Most quantization schemes only run their linear layers in float16 or bfloat16, so requesting float32 fails this check in get_model before any weights are allocated.

Source

Thrown at ChatTTS/model/velocity/model_loader.py:45

    if model_config.quantization is not None:
        quant_config = get_quant_config(
            model_config.quantization,
            model_config.model,
            model_config.hf_config,
            model_config.download_dir,
        )
        capability = torch.cuda.get_device_capability()
        capability = capability[0] * 10 + capability[1]
        if capability < quant_config.get_min_capability():
            raise ValueError(
                f"The quantization method {model_config.quantization} is not "
                "supported for the current GPU. "
                f"Minimum capability: {quant_config.get_min_capability()}. "
                f"Current capability: {capability}."
            )
        supported_dtypes = quant_config.get_supported_act_dtypes()
        if model_config.dtype not in supported_dtypes:
            raise ValueError(
                f"{model_config.dtype} is not supported for quantization "
                f"method {model_config.quantization}. Supported dtypes: "
                f"{supported_dtypes}"
            )
        linear_method = quant_config.get_linear_method()

    with _set_default_torch_dtype(model_config.dtype):
        # Create a model instance.
        # The weights will be initialized as empty tensors.
        with torch.device("cuda"):
            model = LlamaModel(model_config.hf_config, linear_method)
        if model_config.load_format == "dummy":
            # NOTE(woosuk): For accurate performance evaluation, we assign
            # random values to the weights.
            initialize_dummy_weights(model)
        else:
            # Load the weights from the cached or downloaded files.
            model.load_weights(

View on GitHub (pinned to 77b89ee281)

Solutions

Set dtype to float16 (e.g. dtype=torch.float16 or --dtype float16) when using a quantized checkpoint
If on Ampere or newer, try bfloat16 only if it appears in the reported supported_dtypes list
Read the error message: it prints the exact supported dtypes for your quantization method — pick one of those
Disable quantization (quantization=None) if you must run in float32

Example fix

// before
model_config = ModelConfig(..., dtype="float32", quantization="awq")
load_model(model_config)  # ValueError

// after
model_config = ModelConfig(..., dtype="float16", quantization="awq")
load_model(model_config)

Defensive patterns

Strategy: validation

Validate before calling

import torch
from vllm.config import ModelConfig

def check_quant_dtype(model_config: ModelConfig):
    if model_config.quantization is None:
        return
    from vllm.model_executor.weight_utils import get_quant_config
    qc = get_quant_config(model_config.quantization, model_config.model,
                          model_config.hf_config, model_config.download_dir)
    if model_config.dtype not in qc.get_supported_act_dtypes():
        raise ValueError(f"dtype {model_config.dtype} unsupported for "
                         f"{model_config.quantization}; pick one of "
                         f"{qc.get_supported_act_dtypes()}")

Try / catch

try:
    model = load_model(model_config)
except ValueError as e:
    if "is not supported for quantization" in str(e):
        model_config.dtype = torch.float16
        model = load_model(model_config)
    else:
        raise

Prevention

Default quantized runs to dtype float16
Never pass dtype="float32" with AWQ/GPTQ checkpoints
Check the supported_dtypes list printed in the error before retrying

When it happens

Trigger: Constructing a vLLM-style ModelConfig with quantization set (e.g. "awq") and dtype="float32" (or "auto" resolving to an unsupported dtype), then calling load_model/get_model. The check compares model_config.dtype against quant_config.get_supported_act_dtypes().

Common situations: Using an AWQ/GPTQ checkpoint with dtype="float32" or the default torch dtype; older GPUs where "auto" maps to float32 instead of float16; mixing bfloat16 weights with a quant method that only lists float16.

Related errors

AI-assisted analysis of 2noise/ChatTTS@77b89ee281 (2026-08-26). Data as JSON: /api/errors/adb2a71dba882bba. Report an issue: GitHub.