What causes "Unsupported forward dtype: {config.forward_dtype}" in huggingface/transformers?

Set forward_dtype to "mxfp4" or "nvfp4" (exact lowercase) in FPQuantConfig

huggingface/transformers · error · ValueError

Unsupported forward dtype: {config.forward_dtype}

Error message

Unsupported forward dtype: {config.forward_dtype}

What it means

adapt_fp_quant_config converts the user-facing FPQuantConfig into the library-level FPQuantLinearConfig by mapping string dtype names to FPQuantDtype enum members. Only "mxfp4" and "nvfp4" are valid forward dtypes (these are the formats the FPQuant kernel supports for the forward quantization); anything else — including typos, "int4", "fp8" — raises ValueError echoing the offending value.

Source

Thrown at src/transformers/integrations/fp_quant.py:125

            dqweight = torch.nn.Parameter(value)

            return {
                ".dqweight": dqweight,
                # the way the FPQuantLinear module is designed, these parameters are expected in the model
                # even though they are not used so we need to set them to zeros
                ".weight": torch.nn.Parameter(torch.zeros(0)),
                ".qweight": torch.nn.Parameter(torch.zeros(0)),
                ".scales": torch.nn.Parameter(torch.zeros(0)),
            }


def adapt_fp_quant_config(config: FPQuantConfig):
    if config.forward_dtype == "mxfp4":
        forward_dtype = FPQuantDtype.MXFP4
    elif config.forward_dtype == "nvfp4":
        forward_dtype = FPQuantDtype.NVFP4
    else:
        raise ValueError(f"Unsupported forward dtype: {config.forward_dtype}")

    if config.backward_dtype == "bf16":
        backward_dtype = FPQuantDtype.BF16
    elif config.backward_dtype == "mxfp8":
        backward_dtype = FPQuantDtype.MXFP8
    elif config.backward_dtype == "mxfp4":
        backward_dtype = FPQuantDtype.MXFP4
    else:
        raise ValueError(f"Unsupported backward dtype: {config.backward_dtype}")

    return FPQuantLinearConfig(
        forward_dtype=forward_dtype,
        forward_method=config.forward_method,
        backward_dtype=backward_dtype,
        store_master_weights=config.store_master_weights,
        hadamard_group_size=config.hadamard_group_size,
        pseudoquantization=config.pseudoquantization,
        transform_init=config.transform_init,

View on GitHub (pinned to a597f97485)

Solutions

Set forward_dtype to "mxfp4" or "nvfp4" (exact lowercase) in FPQuantConfig
Check the loaded config: print the quantization_config from the checkpoint's config.json and fix the stored string
If the checkpoint legitimately uses a new dtype, upgrade transformers to a version that supports it

Example fix

# before
quant = FPQuantConfig(forward_dtype="nvfp8", backward_dtype="bf16")

# after
quant = FPQuantConfig(forward_dtype="nvfp4", backward_dtype="bf16")

Defensive patterns

Strategy: validation

Validate before calling

VALID_FORWARD = {"mxfp4", "nvfp4"}
assert qc.forward_dtype in VALID_FORWARD, f"forward_dtype must be one of {VALID_FORWARD}, got {qc.forward_dtype!r}"

Type guard

def is_valid_forward_dtype(v: str) -> bool:
    return isinstance(v, str) and v in {"mxfp4", "nvfp4"}

Prevention

Validate quantization dtype strings against the documented enum before constructing the config
Use exact lowercase spellings; the adapter does no normalization

When it happens

Trigger: Constructing FPQuantConfig(forward_dtype=...) or loading a quantized config JSON whose forward_dtype string is not exactly "mxfp4" or "nvfp4", then adapting it (which happens during quantizer setup / model load).

Common situations: Typos in a hand-written quantization config ("mx fp4", "MXFP4" case mismatch — matching is lowercase-exact, "nvfp8"); configs copied from a different quantization ecosystem (torchao/bitsandbytes names); a checkpoint saved with a newer transformers that added a dtype this version does not know.

Related errors

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/b82c67f65a458019. Report an issue: GitHub.