{"record":{"id":"042b7bd28c37bf60","repo":"huggingface/transformers","slug":"unsupported-dtype-dtype-must-be-auto-or-a","errorCode":null,"errorMessage":"Unsupported dtype: '{dtype}'. Must be 'auto' or a valid torch dtype (e.g. 'float16', 'bfloat16').","messagePattern":"Unsupported dtype: '(.+?)'\\. Must be 'auto' or a valid torch dtype \\(e\\.g\\. 'float16', 'bfloat16'\\)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/transformers/cli/serving/model_manager.py","lineNumber":156,"sourceCode":"        self._validate_args()\n\n        # Preloaded models should never be auto-unloaded\n        if force_model is not None:\n            self.model_timeout = -1\n\n        # Preload the forced model after all state is initialized\n        if force_model is not None:\n            self.load_model_and_processor(self.process_model_name(force_model))\n\n    @staticmethod\n    def _resolve_dtype(dtype: str | None):\n        import torch\n\n        if dtype in (\"auto\", None):\n            return dtype\n        resolved = getattr(torch, dtype, None)\n        if not isinstance(resolved, torch.dtype):\n            raise ValueError(\n                f\"Unsupported dtype: '{dtype}'. Must be 'auto' or a valid torch dtype (e.g. 'float16', 'bfloat16').\"\n            )\n        return resolved\n\n    @classmethod\n    def _resolve_attn_implementation(cls, attn_implementation: str | None, device: str | int) -> str | None:\n        r\"\"\"\n        Default to a fast kernel for `mps` when available.\n        \"\"\"\n        if attn_implementation is not None:\n            return attn_implementation\n\n        import torch\n\n        from ...utils.import_utils import is_kernels_available\n\n        is_mps_device = (\n            isinstance(device, str)","sourceCodeStart":138,"sourceCodeEnd":174,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/cli/serving/model_manager.py#L138-L174","documentation":"ModelManager._resolve_dtype maps the --dtype CLI value onto a torch attribute via getattr(torch, dtype) and verifies the result is a torch.dtype instance. 'auto' and None pass through; anything that is not an attribute of torch or not a dtype object (e.g. 'fp16', 'float', 'int') raises ValueError.","triggerScenarios":"Passing --dtype fp16 (abbreviation instead of full name); --dtype float or 'half'; a value like 'auto ' with trailing whitespace; any string that resolves to a non-dtype torch attribute (e.g. 'float' resolves to the builtin-like torch.float which is a dtype, but 'float32_min_normal'-style names would not be).","commonSituations":"Habitual fp16/bf16 shorthand from other frameworks; config files or scripts written for vLLM-style flags; copy-pasting dtype strings from docs of different tools.","solutions":["Use exact torch dtype names: --dtype float16, --dtype bfloat16, --dtype float32, or --dtype auto","Spell abbreviations out (fp16 -> float16, bf16 -> bfloat16)","Strip whitespace/casing issues: values are case-sensitive attribute names on torch"],"exampleFix":"# before\ntransformers serve --model_id gpt2 --dtype fp16   # ValueError\n\n# after\ntransformers serve --model_id gpt2 --dtype float16","handlingStrategy":"validation","validationCode":"import torch\n\nDTYPES = {\"auto\"} | {k for k, v in vars(torch).items() if isinstance(v, torch.dtype)}\nif dtype not in DTYPES:\n    raise SystemExit(f\"Bad dtype {dtype!r}; choose from {sorted(DTYPES)}\")","typeGuard":"import torch\n\ndef is_valid_dtype(value: str) -> bool:\n    if value in (\"auto\", None):\n        return True\n    return isinstance(getattr(torch, value, None), torch.dtype)","tryCatchPattern":null,"preventionTips":["Use full torch dtype names (float16, bfloat16, float32)","Expand shorthands before invoking CLI: fp16->float16, bf16->bfloat16","Validate config-file values with getattr(torch, v, None) checks"],"tags":["serving","dtype","cli","validation","valueerror"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}