hiyouga/LlamaFactory · error · TypeError

compute_dtype must be str or torch.dtype, got {type(self.com

Error message

compute_dtype must be str or torch.dtype, got {type(self.compute_dtype).__name__}.

What it means

BnbParams.compute_dtype accepts either a string (a torch dtype name) or a torch.dtype instance; anything else (int, None, numpy dtype, custom class) raises this TypeError from __post_init__. It is a strict type guard at plugin-parameter parse time, before any model loading happens.

Source

Thrown at src/llamafactory/v1/plugins/model_plugins/quantization.py:55

@dataclass
class BnbParams:
    name: Literal["bnb", "auto"] = "bnb"
    quantization_bit: int | None = None
    compute_dtype: str | Any = "float16"
    double_quantization: bool = True
    quantization_type: str = "nf4"

    def __post_init__(self) -> None:
        import torch

        if isinstance(self.compute_dtype, str):
            dtype = getattr(torch, self.compute_dtype, None)
            if not isinstance(dtype, torch.dtype):
                raise ValueError(f"compute_dtype={self.compute_dtype!r} is not a torch dtype name.")
            self.compute_dtype = dtype
        elif not isinstance(self.compute_dtype, torch.dtype):
            raise TypeError(f"compute_dtype must be str or torch.dtype, got {type(self.compute_dtype).__name__}.")


@QuantizationPlugin("auto").register()
def quantization_auto(
    init_kwargs: dict[str, Any],
    quant_config: dict | BnbParams,
    is_trainable: bool = False,
) -> dict[str, Any]:
    quant_config = QuantizationPlugin.parse_params(quant_config, BnbParams)
    if quant_config.quantization_bit is None:
        logger.warning_rank0("No quantization method applied.")
        return init_kwargs
    if quant_config.quantization_bit not in (4, 8):
        raise ValueError(f"Unsupported quantization bit: {quant_config.quantization_bit} for auto quantization.")

    logger.info_rank0(f"Loading {quant_config.quantization_bit}-bit quantized model.")
    return QuantizationPlugin("bnb")(init_kwargs, quant_config=quant_config, is_trainable=is_trainable)

View on GitHub (pinned to f28afaf635)

Solutions

1. Pass compute_dtype as a string dtype name or torch.float16/torch.bfloat16 object
2. If the value comes from external config, normalize it to a string before constructing the params

Example fix

# before
BnbParams(compute_dtype=16)

# after
BnbParams(compute_dtype="float16")  # or torch.float16

import torch
assert isinstance(compute_dtype, (str, torch.dtype)), f"compute_dtype must be str or torch.dtype, got {type(compute_dtype).__name__}"

def is_valid_compute_dtype(v) -> bool:
    import torch
    return isinstance(v, torch.dtype) or (isinstance(v, str) and isinstance(getattr(torch, v, None), torch.dtype))

When it happens

Trigger: Programmatically building the quantization config dict with compute_dtype=16, compute_dtype=None, or a numpy dtype instead of str/torch.dtype.

Common situations: Passing a numeric precision indicator from another config schema; forgetting that the field is a dtype, not a bit-width; YAML auto-parsing oddities where the value is not a plain string.

Related errors

• compute_dtype={self.compute_dtype!r} is not a torch dtype na
• Bitsandbytes only accepts 4-bit or 8-bit quantization.
• Unsupported quantization bit: {quant_config.quantization_bit
• Bitsandbytes only accepts 4-bit or 8-bit quantization.
• Only 4-bit quantized model can use fsdp+qlora or auto device

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/1a043ac65da20eeb. Report an issue: GitHub.