{"record":{"id":"f368aa1bfcc809fe","repo":"vllm-project/vllm","slug":"the-quantization-method-model-config-quantization","errorCode":null,"errorMessage":"The quantization method {model_config.quantization} is not supported for the current GPU. Minimum capability: {quant_config.get_min_capability()}. Current capability: {capability}.","messagePattern":"The quantization method (.+?) is not supported for the current GPU\\. Minimum capability: (.+?)\\. Current capability: (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/config/vllm.py","lineNumber":774,"sourceCode":"            enable_trace_function_call(log_path)\n\n    @staticmethod\n    def _get_quantization_config(\n        model_config: ModelConfig, load_config: LoadConfig\n    ) -> QuantizationConfig | None:\n        \"\"\"Get the quantization config.\"\"\"\n        from vllm.platforms import current_platform\n\n        if model_config.quantization is not None:\n            from vllm.model_executor.model_loader.weight_utils import get_quant_config\n\n            quant_config = get_quant_config(model_config, load_config)\n            capability_tuple = current_platform.get_device_capability()\n\n            if capability_tuple is not None:\n                capability = capability_tuple.to_int()\n                if capability < quant_config.get_min_capability():\n                    raise ValueError(\n                        f\"The quantization method {model_config.quantization} \"\n                        \"is not supported for the current GPU. Minimum \"\n                        f\"capability: {quant_config.get_min_capability()}. \"\n                        f\"Current capability: {capability}.\"\n                    )\n            supported_dtypes = quant_config.get_supported_act_dtypes()\n            if model_config.dtype not in supported_dtypes:\n                raise ValueError(\n                    f\"{model_config.dtype} is not supported for quantization \"\n                    f\"method {model_config.quantization}. Supported dtypes: \"\n                    f\"{supported_dtypes}\"\n                )\n            quant_config.maybe_update_config(\n                model_config.model,\n                hf_config=model_config.hf_config,\n                revision=model_config.revision,\n            )\n            return quant_config","sourceCodeStart":756,"sourceCodeEnd":792,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/config/vllm.py#L756-L792","documentation":"During quantization config resolution, vLLM fetches the model's quant config, queries the current platform's device capability, and compares it against quant_config.get_min_capability(). If the GPU's compute capability (as an int, e.g. 80 for SM80, 90 for SM90) is below the quantization method's minimum, it raises this error. For example, FP8 or certain AWQ/Marlin variants require SM80+ or SM89+, and running them on older GPUs produces undefined kernels, so vLLM refuses to start.","triggerScenarios":"Loading a quantized checkpoint (e.g. compressed-tensors FP8, AWQ marlin, GPTQ marlin) on a GPU whose capability is below the method's minimum: FP8 checkpoints on pre-Ampere cards, some FP8 variants on Ampere (SM80/86) when min is 89, etc. Raised from get_quant_config path inside VllmConfig when model_config.quantization is set and current_platform.get_device_capability() returns a tuple.","commonSituations":"Downloading an FP8-quantized community model (most are FP8 w8a8) and running it on a 3090/A10/A100 that lacks required capability; migrating between GPU generations; using quantization='fp8' explicitly on T4/V100 hardware; headless CI machines with older GPUs.","solutions":["Use a checkpoint quantized for your GPU class (e.g. GPTQ/AWQ int4 for pre-FP8 GPUs) or an unquantized BF16/FP16 checkpoint","Run on hardware meeting the minimum capability stated in the message (e.g. H100/Ada for FP8 methods requiring >=89/90)","Check current capability first: torch.cuda.get_device_capability() and compare against the method's documented minimum before launching","Set model_config.quantization to a method supported by your card or leave auto-detection to pick a compatible kernel"],"exampleFix":"# before\nllm = LLM(model=\"red-panda/...-fp8\")  # on RTX 3090 (SM86 < min 89)\n# after\nllm = LLM(model=\"...-awq-int4\")  # AWQ works on SM86\n# or run on H100/Ada (SM89/90+)","handlingStrategy":"validation","validationCode":"import torch\ncap = torch.cuda.get_device_capability(0)\ncap_int = cap[0] * 10 + cap[1]\nMIN_CAP = {\"fp8\": 89, \"awq_marlin\": 80}  # per method; consult quant_config\nassert cap_int >= MIN_CAP.get(quant_method, 0), \\\n    f\"GPU capability {cap_int} below minimum for {quant_method}\"","typeGuard":"def gpu_supports(cap_int: int, min_cap: int) -> bool:\n    return cap_int >= min_cap","tryCatchPattern":"try:\n    engine = LLM(model=ckpt)\nexcept ValueError as e:\n    if \"Minimum capability\" in str(e):\n        raise SystemExit(f\"GPU too old for {ckpt}; pick a GPTQ/AWQ or bf16 checkpoint\")\n    raise","preventionTips":["Check torch.cuda.get_device_capability() against the quant method's minimum before pulling large checkpoints","Prefer GPTQ/AWQ int4 checkpoints for pre-Ampere or Ampere consumer cards","Automate checkpoint selection per node GPU class in deployment tooling"],"tags":["quantization","gpu-capability","hardware","config"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}