huggingface/transformers · error · ValueError

Cannot reshape tensor with shape {tensor.shape} into {target

Error message

Cannot reshape tensor with shape {tensor.shape} into {target_shape}

What it means

Raised by LinearToConv3d.convert (core_model_loading.py:405). The op reshapes a flattened [out_features, in_features*kH*kW*kD] Linear weight back into the Conv3d layout [out_ch, in_ch, kH, kW, kD] using the configured in_channels and kernel_size. It first checks total element counts: tensor.numel() must equal out_ch * in_channels * prod(kernel_size). A mismatch means the (in_channels, kernel_size) configuration does not describe this checkpoint tensor, so the reshape is impossible.

Source

Thrown at src/transformers/core_model_loading.py:405

class LinearToConv3d(ConversionOps):
    """Flattened Linear weights → Conv3d layout."""

    def __init__(self, in_channels: int, kernel_size: tuple[int, int, int]):
        self.in_channels = in_channels
        self.kernel_size = kernel_size

    @torch.no_grad
    def convert(
        self, input_dict: dict[str, torch.Tensor], source_patterns: list[str], target_patterns: list[str], **kwargs
    ) -> dict[str, torch.Tensor]:
        target_pattern = Conv3dToLinear._get_target_pattern(input_dict, source_patterns, target_patterns)
        tensors = next(iter(input_dict.values()))
        tensor = tensors[0] if isinstance(tensors, list) else tensors

        target_shape = (tensor.shape[0], self.in_channels, *self.kernel_size)
        if tensor.numel() != math.prod(target_shape):
            raise ValueError(f"Cannot reshape tensor with shape {tensor.shape} into {target_shape}")

        return {target_pattern: tensor.reshape(target_shape).contiguous()}

    @property
    def reverse_op(self) -> ConversionOps:
        return Conv3dToLinear(in_channels=self.in_channels, kernel_size=self.kernel_size)


class PermuteForRope(ConversionOps):
    """
    Applies the permutation required to convert complex RoPE weights to the split sin/cos format.
    """

    def __init__(
        self, subconfig_key: str | None = None, permute_layer_names: list[str] | None = None, inverse: bool = False
    ):
        self.subconfig_key = subconfig_key
        self.inverse = inverse

View on GitHub (pinned to a597f97485)

Solutions

1. Compute tensor.shape[1] from the checkpoint and derive the correct factorization: in_channels * prod(kernel_size) must equal it (e.g. 20736 = 3*3*3*768 for kernel 3^3 and 768 channels).
2. Update the LinearToConv3d arguments in the conversion recipe to the correct in_channels/kernel_size.
3. If saving (reverse direction), make sure the forward op Conv3dToLinear was configured with the same geometry so the round trip is consistent.

Example fix

# before: wrong kernel geometry
LinearToConv3d(in_channels=768, kernel_size=(2, 2, 2))

# after: 768 * 3*3*3 = 20736 matches tensor.shape[1]
LinearToConv3d(in_channels=768, kernel_size=(3, 3, 3))

import math
expected_cols = in_channels * math.prod(kernel_size)
assert tensor.shape[1] == expected_cols, (
    f'Linear weight has {tensor.shape[1]} cols but in_channels*prod(kernel_size)={expected_cols}; '
    'fix in_channels/kernel_size in LinearToConv3d'
)

def fits_conv3d_layout(tensor: "torch.Tensor", in_channels: int, kernel_size: tuple[int, ...]) -> bool:
    import math
    return tensor.ndim == 2 and tensor.shape[1] == in_channels * math.prod(kernel_size)

When it happens

Trigger: Constructing LinearToConv3d(in_channels=..., kernel_size=(kH,kW,kD)) where in_channels * kH * kW * kD != tensor.shape[1] of the collected Linear weight — e.g. declaring kernel_size=(3,3,3) and in_channels=768 for a tensor of shape [out, 20736] (which implies a different product).

Common situations: Reverse-converting (saving) a model whose original implementation used a Conv3d with a patch size or channel count different from what the conversion recipe hard-codes; common when porting video encoders where patch embedding is expressed as a Linear (patchify) layer and the recipe author guessed the wrong kernel geometry.

Related errors

• Conv3dToLinear expects a 5D or 2D tensor, got {tensor.ndim}D
• `num_head` was provided as a list of length {len(num_heads)}
• `head_dim` was provided as a list of length {len(num_heads)}
• Failed to convert {kwargs.get('full_layer_name')}
• PermuteForRope expects a single tensor per key.

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/37e977a5b907192b. Report an issue: GitHub.