{"record":{"id":"b82c67f65a458019","repo":"huggingface/transformers","slug":"unsupported-forward-dtype-config-forward-dtype","errorCode":null,"errorMessage":"Unsupported forward dtype: {config.forward_dtype}","messagePattern":"Unsupported forward dtype: (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/transformers/integrations/fp_quant.py","lineNumber":125,"sourceCode":"            dqweight = torch.nn.Parameter(value)\n\n            return {\n                \".dqweight\": dqweight,\n                # the way the FPQuantLinear module is designed, these parameters are expected in the model\n                # even though they are not used so we need to set them to zeros\n                \".weight\": torch.nn.Parameter(torch.zeros(0)),\n                \".qweight\": torch.nn.Parameter(torch.zeros(0)),\n                \".scales\": torch.nn.Parameter(torch.zeros(0)),\n            }\n\n\ndef adapt_fp_quant_config(config: FPQuantConfig):\n    if config.forward_dtype == \"mxfp4\":\n        forward_dtype = FPQuantDtype.MXFP4\n    elif config.forward_dtype == \"nvfp4\":\n        forward_dtype = FPQuantDtype.NVFP4\n    else:\n        raise ValueError(f\"Unsupported forward dtype: {config.forward_dtype}\")\n\n    if config.backward_dtype == \"bf16\":\n        backward_dtype = FPQuantDtype.BF16\n    elif config.backward_dtype == \"mxfp8\":\n        backward_dtype = FPQuantDtype.MXFP8\n    elif config.backward_dtype == \"mxfp4\":\n        backward_dtype = FPQuantDtype.MXFP4\n    else:\n        raise ValueError(f\"Unsupported backward dtype: {config.backward_dtype}\")\n\n    return FPQuantLinearConfig(\n        forward_dtype=forward_dtype,\n        forward_method=config.forward_method,\n        backward_dtype=backward_dtype,\n        store_master_weights=config.store_master_weights,\n        hadamard_group_size=config.hadamard_group_size,\n        pseudoquantization=config.pseudoquantization,\n        transform_init=config.transform_init,","sourceCodeStart":107,"sourceCodeEnd":143,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/integrations/fp_quant.py#L107-L143","documentation":"adapt_fp_quant_config converts the user-facing FPQuantConfig into the library-level FPQuantLinearConfig by mapping string dtype names to FPQuantDtype enum members. Only \"mxfp4\" and \"nvfp4\" are valid forward dtypes (these are the formats the FPQuant kernel supports for the forward quantization); anything else — including typos, \"int4\", \"fp8\" — raises ValueError echoing the offending value.","triggerScenarios":"Constructing FPQuantConfig(forward_dtype=...) or loading a quantized config JSON whose forward_dtype string is not exactly \"mxfp4\" or \"nvfp4\", then adapting it (which happens during quantizer setup / model load).","commonSituations":"Typos in a hand-written quantization config (\"mx fp4\", \"MXFP4\" case mismatch — matching is lowercase-exact, \"nvfp8\"); configs copied from a different quantization ecosystem (torchao/bitsandbytes names); a checkpoint saved with a newer transformers that added a dtype this version does not know.","solutions":["Set forward_dtype to \"mxfp4\" or \"nvfp4\" (exact lowercase) in FPQuantConfig","Check the loaded config: print the quantization_config from the checkpoint's config.json and fix the stored string","If the checkpoint legitimately uses a new dtype, upgrade transformers to a version that supports it"],"exampleFix":"# before\nquant = FPQuantConfig(forward_dtype=\"nvfp8\", backward_dtype=\"bf16\")\n\n# after\nquant = FPQuantConfig(forward_dtype=\"nvfp4\", backward_dtype=\"bf16\")","handlingStrategy":"validation","validationCode":"VALID_FORWARD = {\"mxfp4\", \"nvfp4\"}\nassert qc.forward_dtype in VALID_FORWARD, f\"forward_dtype must be one of {VALID_FORWARD}, got {qc.forward_dtype!r}\"","typeGuard":"def is_valid_forward_dtype(v: str) -> bool:\n    return isinstance(v, str) and v in {\"mxfp4\", \"nvfp4\"}","tryCatchPattern":null,"preventionTips":["Validate quantization dtype strings against the documented enum before constructing the config","Use exact lowercase spellings; the adapter does no normalization"],"tags":["fp-quant","quantization","config","dtype","validation"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}