hiyouga/LlamaFactory · error · ValueError

compute_dtype={self.compute_dtype!r} is not a torch dtype na

Error message

compute_dtype={self.compute_dtype!r} is not a torch dtype name.

What it means

BnbParams.__post_init__ converts the compute_dtype string to a torch.dtype via getattr(torch, name). If the string is not the name of a torch dtype (misspelled, wrong casing, or an unrelated torch attribute that is not a dtype), conversion fails and this ValueError is raised. Valid values are strings like 'float16', 'bfloat16', 'float32'.

Source

Thrown at src/llamafactory/v1/plugins/model_plugins/quantization.py:52

    ) -> dict[str, Any]:
        return super().__call__(init_kwargs, quant_config=quant_config, is_trainable=is_trainable)


@dataclass
class BnbParams:
    name: Literal["bnb", "auto"] = "bnb"
    quantization_bit: int | None = None
    compute_dtype: str | Any = "float16"
    double_quantization: bool = True
    quantization_type: str = "nf4"

    def __post_init__(self) -> None:
        import torch

        if isinstance(self.compute_dtype, str):
            dtype = getattr(torch, self.compute_dtype, None)
            if not isinstance(dtype, torch.dtype):
                raise ValueError(f"compute_dtype={self.compute_dtype!r} is not a torch dtype name.")
            self.compute_dtype = dtype
        elif not isinstance(self.compute_dtype, torch.dtype):
            raise TypeError(f"compute_dtype must be str or torch.dtype, got {type(self.compute_dtype).__name__}.")


@QuantizationPlugin("auto").register()
def quantization_auto(
    init_kwargs: dict[str, Any],
    quant_config: dict | BnbParams,
    is_trainable: bool = False,
) -> dict[str, Any]:
    quant_config = QuantizationPlugin.parse_params(quant_config, BnbParams)
    if quant_config.quantization_bit is None:
        logger.warning_rank0("No quantization method applied.")
        return init_kwargs
    if quant_config.quantization_bit not in (4, 8):
        raise ValueError(f"Unsupported quantization bit: {quant_config.quantization_bit} for auto quantization.")

View on GitHub (pinned to f28afaf635)

Solutions

1. Use the exact torch dtype name: float16, bfloat16, or float32
2. Check casing: 'Float16' is invalid
3. Alternatively pass an actual torch.dtype object if constructing BnbParams programmatically

Example fix

# before
quantization:
  compute_dtype: fp16

# after
quantization:
  compute_dtype: float16

import torch
if isinstance(compute_dtype, str):
    assert isinstance(getattr(torch, compute_dtype, None), torch.dtype), f"bad compute_dtype {compute_dtype!r}"

def is_torch_dtype_name(s: str) -> bool:
    import torch
    return isinstance(getattr(torch, s, None), torch.dtype)

When it happens

Trigger: Setting compute_dtype: fp16 / bf16 / float / float64-typo in the quantization config; anything where getattr(torch, s) is not a torch.dtype instance.

Common situations: Users abbreviate dtypes (fp16, bf16) out of habit from other frameworks; or copy a config from a tool that uses different dtype names.

Related errors

• compute_dtype must be str or torch.dtype, got {type(self.com
• Unsupported quantization bit: {quant_config.quantization_bit
• Bitsandbytes only accepts 4-bit or 8-bit quantization.
• Bitsandbytes only accepts 4-bit or 8-bit quantization.
• Only 4-bit quantized model can use fsdp+qlora or auto device

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/4a890173596a14ca. Report an issue: GitHub.