Comfy-Org/ComfyUI · error · ValueError

MXFP8 requires 2D tensor, got {tensor.dim()}D

Error message

MXFP8 requires 2D tensor, got {tensor.dim()}D

What it means

TensorCoreMXFP8Layout.quantize in comfy/quant_ops.py rejects any tensor that is not exactly 2D. MXFP8 stores block scales on a 32x32 2D grid over (out_channels, in_channels), so 1D biases, 3D conv weights, or 4D tensors have no valid block layout and quantization fails fast rather than silently mis-shaping scales.

Source

Thrown at comfy/quant_ops.py:153

        if stochastic_rounding > 0:
            if inplace_ops:
                tensor *= (1.0 / scale).to(tensor.dtype)
            else:
                tensor = tensor * (1.0 / scale).to(tensor.dtype)
            qdata = comfy.float.stochastic_rounding(tensor, dtype=cls.FP8_DTYPE, seed=stochastic_rounding)
        else:
            qdata = ck.quantize_per_tensor_fp8(tensor, scale, cls.FP8_DTYPE)

        params = cls.Params(scale=scale.float(), orig_dtype=orig_dtype, orig_shape=orig_shape)
        return qdata, params


class TensorCoreMXFP8Layout(_CKMxfp8Layout):
    @classmethod
    def quantize(cls, tensor, scale=None, stochastic_rounding=0, inplace_ops=False):
        if tensor.dim() != 2:
            raise ValueError(f"MXFP8 requires 2D tensor, got {tensor.dim()}D")

        orig_dtype = tensor.dtype
        orig_shape = tuple(tensor.shape)

        padded_shape = cls.get_padded_shape(orig_shape)
        needs_padding = padded_shape != orig_shape

        if stochastic_rounding > 0:
            qdata, block_scale = comfy.float.stochastic_round_quantize_mxfp8_by_block(tensor, pad_32x=needs_padding, seed=stochastic_rounding)
        else:
            qdata, block_scale = ck.quantize_mxfp8(tensor, pad_32x=needs_padding)

        params = cls.Params(
            scale=block_scale,
            orig_dtype=orig_dtype,
            orig_shape=orig_shape,
        )
        return qdata, params

View on GitHub (pinned to 1c6d8d45b3)

Solutions

1. Only route 2D nn.Linear weight matrices to the MXFP8 quantizer; skip norms, biases, embeddings and conv weights.
2. If quantizing a conv, first reshape to 2D ([out, in*kh*kw]) only if your kernel path supports dequantizing back to 4D.
3. Check for accidental unsqueeze/squeeze upstream that changed rank before quantize() was called.

Example fix

# before
TensorCoreMXFP8Layout.quantize(conv.weight)  # 4D -> ValueError

# after
for name, module in model.named_modules():
    if isinstance(module, torch.nn.Linear):
        q, p = TensorCoreMXFP8Layout.quantize(module.weight)

if tensor.dim() != 2:
    raise ValueError(f'skip MXFP8 for {tensor.dim()}D tensor; quantize only Linear weights')

def is_mxfp8_quantizable(t: torch.Tensor) -> bool:
    return isinstance(t, torch.Tensor) and t.dim() == 2 and t.is_floating_point()

When it happens

Trigger: Calling TensorCoreMXFP8Layout.quantize on a Conv weight (4D), an embedding/norm tensor (1D/3D), or any tensor produced after an unsolicited reshape; quantizing a model whose Linear weights were transposed to 3D by an adapter.

Common situations: Model-quantization scripts that iterate all parameters instead of only nn.Linear weights; trying to apply MXFP8 to conv-heavy architectures; passing a fused/qkv tensor with an extra head dimension.

Related errors

• Missing MXFP8 block scales for layer {layer_name}
• NVFP4 requires 2D tensor, got {tensor.dim()}D
• Unsupported dtype
• Unexpected token width: {out_x.shape[-1]}
• Invalid affine shape: {ext.shape}

AI-assisted analysis of Comfy-Org/ComfyUI@1c6d8d45b3 (2026-08-14). Data as JSON: /api/errors/b3f214f09c0482d4. Report an issue: GitHub.