Comfy-Org/ComfyUI · error · ValueError

NVFP4 requires 2D tensor, got {tensor.dim()}D

Error message

NVFP4 requires 2D tensor, got {tensor.dim()}D

What it means

TensorCoreNVFP4Layout.quantize in comfy/quant_ops.py raises when the input tensor is not 2D. NVFP4 uses a 2D block-scale grid (typically 16-element groups over a flattened [N, K] matrix); a tensor of any other rank cannot be assigned a valid block layout, so quantization is rejected up front.

Source

Thrown at comfy/quant_ops.py:178

        if stochastic_rounding > 0:
            qdata, block_scale = comfy.float.stochastic_round_quantize_mxfp8_by_block(tensor, pad_32x=needs_padding, seed=stochastic_rounding)
        else:
            qdata, block_scale = ck.quantize_mxfp8(tensor, pad_32x=needs_padding)

        params = cls.Params(
            scale=block_scale,
            orig_dtype=orig_dtype,
            orig_shape=orig_shape,
        )
        return qdata, params


class TensorCoreNVFP4Layout(_CKNvfp4Layout):
    @classmethod
    def quantize(cls, tensor, scale=None, stochastic_rounding=0, inplace_ops=False):
        if tensor.dim() != 2:
            raise ValueError(f"NVFP4 requires 2D tensor, got {tensor.dim()}D")

        orig_dtype = tensor.dtype
        orig_shape = tuple(tensor.shape)

        if scale is None or (isinstance(scale, str) and scale == "recalculate"):
            scale = torch.amax(tensor.abs()) / (ck.float_utils.F8_E4M3_MAX * ck.float_utils.F4_E2M1_MAX)

        if not isinstance(scale, torch.Tensor):
            scale = torch.tensor(scale)
        scale = scale.to(device=tensor.device, dtype=torch.float32)

        padded_shape = cls.get_padded_shape(orig_shape)
        needs_padding = padded_shape != orig_shape

        if stochastic_rounding > 0:
            qdata, block_scale = comfy.float.stochastic_round_quantize_nvfp4_by_block(tensor, scale, pad_16x=needs_padding, seed=stochastic_rounding)
        else:
            qdata, block_scale = ck.quantize_nvfp4(tensor, scale, pad_16x=needs_padding)

View on GitHub (pinned to 1c6d8d45b3)

Solutions

1. Restrict NVFP4 quantization to 2D nn.Linear weight matrices.
2. Guard the call: skip tensors with tensor.dim() != 2 and leave them in higher precision.
3. Reshape 4D conv weights to [out, in*kh*kw] only if the corresponding dequantize path restores the original shape via orig_shape.

Example fix

# before
qdata, params = TensorCoreNVFP4Layout.quantize(weight)  # crashes on 4D

# after
if weight.dim() == 2:
    qdata, params = TensorCoreNVFP4Layout.quantize(weight)
else:
    qdata, params = weight, None  # keep unquantized

if tensor.dim() != 2:
    raise ValueError(f'skip NVFP4 for {tensor.dim()}D tensor; quantize only Linear weights')

def is_nvfp4_quantizable(t: torch.Tensor) -> bool:
    return isinstance(t, torch.Tensor) and t.dim() == 2 and t.is_floating_point()

When it happens

Trigger: Passing a Conv weight (4D), norm/bias vector (1D), or embedding tensor to TensorCoreNVFP4Layout.quantize; quantizing weights that an adapter reshaped to 3D; blanket quantization loops that hit every parameter.

Common situations: Quantization scripts that walk all model parameters instead of Linear layers only; attempting NVFP4 on CNNs or models with fused attention weights carrying extra dims; tensors with a leftover batch/head dimension.

Related errors

• Missing NVFP4 scales for layer {layer_name}
• MXFP8 requires 2D tensor, got {tensor.dim()}D
• Unsupported dtype
• Unexpected token width: {out_x.shape[-1]}
• Invalid affine shape: {ext.shape}

AI-assisted analysis of Comfy-Org/ComfyUI@1c6d8d45b3 (2026-08-14). Data as JSON: /api/errors/b7f19ad73feef60f. Report an issue: GitHub.