{"record":{"id":"1a043ac65da20eeb","repo":"hiyouga/LlamaFactory","slug":"compute-dtype-must-be-str-or-torch-dtype-got-typ","errorCode":null,"errorMessage":"compute_dtype must be str or torch.dtype, got {type(self.compute_dtype).__name__}.","messagePattern":"compute_dtype must be str or torch\\.dtype, got (.+?)\\.","errorType":"validation","errorClass":"TypeError","httpStatus":null,"severity":"error","filePath":"src/llamafactory/v1/plugins/model_plugins/quantization.py","lineNumber":55,"sourceCode":"\n@dataclass\nclass BnbParams:\n    name: Literal[\"bnb\", \"auto\"] = \"bnb\"\n    quantization_bit: int | None = None\n    compute_dtype: str | Any = \"float16\"\n    double_quantization: bool = True\n    quantization_type: str = \"nf4\"\n\n    def __post_init__(self) -> None:\n        import torch\n\n        if isinstance(self.compute_dtype, str):\n            dtype = getattr(torch, self.compute_dtype, None)\n            if not isinstance(dtype, torch.dtype):\n                raise ValueError(f\"compute_dtype={self.compute_dtype!r} is not a torch dtype name.\")\n            self.compute_dtype = dtype\n        elif not isinstance(self.compute_dtype, torch.dtype):\n            raise TypeError(f\"compute_dtype must be str or torch.dtype, got {type(self.compute_dtype).__name__}.\")\n\n\n@QuantizationPlugin(\"auto\").register()\ndef quantization_auto(\n    init_kwargs: dict[str, Any],\n    quant_config: dict | BnbParams,\n    is_trainable: bool = False,\n) -> dict[str, Any]:\n    quant_config = QuantizationPlugin.parse_params(quant_config, BnbParams)\n    if quant_config.quantization_bit is None:\n        logger.warning_rank0(\"No quantization method applied.\")\n        return init_kwargs\n    if quant_config.quantization_bit not in (4, 8):\n        raise ValueError(f\"Unsupported quantization bit: {quant_config.quantization_bit} for auto quantization.\")\n\n    logger.info_rank0(f\"Loading {quant_config.quantization_bit}-bit quantized model.\")\n    return QuantizationPlugin(\"bnb\")(init_kwargs, quant_config=quant_config, is_trainable=is_trainable)\n","sourceCodeStart":37,"sourceCodeEnd":73,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/src/llamafactory/v1/plugins/model_plugins/quantization.py#L37-L73","documentation":"BnbParams.compute_dtype accepts either a string (a torch dtype name) or a torch.dtype instance; anything else (int, None, numpy dtype, custom class) raises this TypeError from __post_init__. It is a strict type guard at plugin-parameter parse time, before any model loading happens.","triggerScenarios":"Programmatically building the quantization config dict with compute_dtype=16, compute_dtype=None, or a numpy dtype instead of str/torch.dtype.","commonSituations":"Passing a numeric precision indicator from another config schema; forgetting that the field is a dtype, not a bit-width; YAML auto-parsing oddities where the value is not a plain string.","solutions":["Pass compute_dtype as a string dtype name or torch.float16/torch.bfloat16 object","If the value comes from external config, normalize it to a string before constructing the params"],"exampleFix":"# before\nBnbParams(compute_dtype=16)\n\n# after\nBnbParams(compute_dtype=\"float16\")  # or torch.float16","handlingStrategy":"type-guard","validationCode":"import torch\nassert isinstance(compute_dtype, (str, torch.dtype)), f\"compute_dtype must be str or torch.dtype, got {type(compute_dtype).__name__}\"","typeGuard":"def is_valid_compute_dtype(v) -> bool:\n    import torch\n    return isinstance(v, torch.dtype) or (isinstance(v, str) and isinstance(getattr(torch, v, None), torch.dtype))","tryCatchPattern":null,"preventionTips":["Never pass numeric bit-widths as compute_dtype","Centralize dtype parsing in one helper for programmatic configs"],"tags":["quantization","bitsandbytes","dtype","type-error"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}