{"record":{"id":"85caa527ee4ee24e","repo":"vllm-project/vllm","slug":"the-model-type-model-type-r-does-not-support-flo","errorCode":null,"errorMessage":"The model type {model_type!r} does not support float16. Reason: {reason}","messagePattern":"The model type (.+?) does not support float16\\. Reason: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/config/model.py","lineNumber":2205,"sourceCode":"_FLOAT16_NOT_SUPPORTED_MODELS = {\n    \"gemma2\": \"Numerical instability. Please use bfloat16 or float32 instead.\",\n    \"gemma3\": \"Numerical instability. Please use bfloat16 or float32 instead.\",\n    \"gemma3_text\": \"Numerical instability. Please use bfloat16 or float32 instead.\",\n    \"glm4\": \"Numerical instability. Please use bfloat16 or float32 instead.\",\n}\n\n\ndef _is_valid_dtype(model_type: str, dtype: torch.dtype):\n    if model_type in _FLOAT16_NOT_SUPPORTED_MODELS and dtype == torch.float16:  # noqa: E501, SIM103\n        return False\n\n    return True\n\n\ndef _check_valid_dtype(model_type: str, dtype: torch.dtype):\n    if model_type in _FLOAT16_NOT_SUPPORTED_MODELS and dtype == torch.float16:\n        reason = _FLOAT16_NOT_SUPPORTED_MODELS[model_type]\n        raise ValueError(\n            f\"The model type {model_type!r} does not support float16. Reason: {reason}\"\n        )\n\n    return True\n\n\ndef _resolve_auto_dtype(\n    model_type: str,\n    config_dtype: torch.dtype,\n    *,\n    is_pooling_model: bool,\n):\n    supported_dtypes = [\n        dtype\n        for dtype in current_platform.supported_dtypes\n        if _is_valid_dtype(model_type, dtype)\n    ]\n","sourceCodeStart":2187,"sourceCodeEnd":2223,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/config/model.py#L2187-L2223","documentation":"_check_valid_dtype raises when the model type appears in _FLOAT16_NOT_SUPPORTED_MODELS (with a per-model reason) and float16 was requested. Certain architectures (numerically unstable in fp16, e.g. some Gemma/Nemotron variants) are hard-blocked from float16 regardless of hardware.","triggerScenarios":"Passing dtype='float16' (or a config whose torch_dtype resolves to fp16) for a model_type listed in _FLOAT16_NOT_SUPPORTED_MODELS; raised from _get_and_verify_dtype during ModelConfig construction.","commonSituations":"Forcing fp16 on V100-class GPUs (no bf16) for a model family that overflows in fp16; fine-tuned/community checkpoints that defaulted torch_dtype to float16 in config.json.","solutions":["Use bfloat16 (dtype='bfloat16') or auto for the model.","Use a variant of the model trained/tuned for fp16 if the vendor provides one.","If config.json sets torch_dtype: float16, override at launch with --dtype bfloat16."],"exampleFix":"# before\nvllm serve google/gemma-2-9b --dtype float16\n# after\nvllm serve google/gemma-2-9b --dtype bfloat16","handlingStrategy":"validation","validationCode":"from vllm.config.model import _FLOAT16_NOT_SUPPORTED_MODELS\ndef dtype_allowed(model_type: str, dtype) -> bool:\n    return not (model_type in _FLOAT16_NOT_SUPPORTED_MODELS and str(dtype).endswith('float16'))\n# default to bfloat16/auto for listed model types","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Default to --dtype auto; only pin float16 on hardware without bf16 and for models known to be fp16-stable.","Check _FLOAT16_NOT_SUPPORTED_MODELS (or release notes) when adding new architectures to your fleet.","Fix checkpoints whose config.json hardcodes torch_dtype: float16 for blocked model types."],"tags":["dtype","float16","model-support","config"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}