huggingface/transformers · error · ValueError

PermuteForRope expects a single tensor per key.

Error message

PermuteForRope expects a single tensor per key.

What it means

Raised by PermuteForRope.convert (core_model_loading.py:462). This op permutes q/k weight matrices to move between complex-RoPE layout and split sin/cos layout; the permutation is only defined for a single 2D weight per key. If a key matched by permute_layer_names arrives as a list holding zero or more than one tensor (e.g. because several source checkpoint keys were fused/stacked into that target key), the op cannot know which tensor to permute and aborts.

Source

Thrown at src/transformers/core_model_loading.py:462

    def convert(
        self,
        input_dict: dict[str, list[torch.Tensor] | torch.Tensor],
        source_patterns: list[str],
        target_patterns: list[str],
        config,
        **kwargs,
    ) -> dict[str, list[torch.Tensor]]:
        self.config = config
        output: dict[str, list[torch.Tensor]] = {}
        for key, tensors in input_dict.items():
            # Permute q and key weights back (skip biases) to match original RoPE implementation
            if not any(name in key for name in self.permute_layer_names):
                output[key] = tensors
                continue

            if isinstance(tensors, list):
                if len(tensors) != 1:
                    raise ValueError("PermuteForRope expects a single tensor per key.")
                tensors = tensors[0]
            output[key] = self._apply(tensors)
        return output

    @property
    def reverse_op(self) -> ConversionOps:
        return PermuteForRope(
            subconfig_key=self.subconfig_key, permute_layer_names=self.permute_layer_names, inverse=not self.inverse
        )


class VisionFuseAndPermuteForRope(ConversionOps):
    """
    Applies the permutation required to convert complex RoPE weights to the split sin/cos format on fused QKV.
    Same as calling `PermuteForRope() + Concatenate()` but lets us call `Permute` only on a subset of chunked tensors.

    NOTE: this conversion applies only to a vision backbone in multimodal models, because it checks `config.vision_config`
    """

View on GitHub (pinned to a597f97485)

Solutions

1. Debug what input_dict contains for the failing key: if a list, find out which prior op fused/sharded it and adjust ordering or patterns so PermuteForRope sees one plain tensor per key.
2. Restrict permute_layer_names so it only matches actual q/k weight names (e.g. 'q_proj' and 'k_proj'), not keys that carry tensor lists.
3. If shards are involved, merge/concatenate them before the permute step in the operations chain.

Example fix

# before: pattern also matches fused list tensors
PermuteForRope(subconfig_key="text_config", permute_layer_names=["q", "k", "gate"])

# after: only single-tensor q/k weights
PermuteForRope(subconfig_key="text_config", permute_layer_names=["q_proj", "k_proj"])

for key, tensors in collected.items():
    if any(name in key for name in permute_layer_names):
        assert not isinstance(tensors, list) or len(tensors) == 1, (
            f'PermuteForRope key {key} carries {len(tensors)} tensors; expected 1'
        )

def is_single_tensor_entry(entry) -> bool:
    return (not isinstance(entry, list)) or len(entry) == 1

When it happens

Trigger: A PermuteForRope(subconfig_key=..., permute_layer_names=[...]) op where one of the matched keys maps to a list of tensors with len != 1 — typically when an earlier op in the chain (e.g. Fuse) produced stacked tensors under that key, or when the rename layer targets both a weight and something else under one key.

Common situations: Conversion recipes for Llama-family / RoPE-based models (or vision towers using RoPE, e.g. Qwen-VL style) where q_proj/k_proj keys are expected single tensors, but the checkpoint or a preceding conversion step delivered fused or multiple shards (e.g. multi-GPU sharded checkpoints collected under one key).

Related errors

• Failed to convert {kwargs.get('full_layer_name')}
• Conv3dToLinear expects a 5D or 2D tensor, got {tensor.ndim}D
• Expected pattern {key} in collected tensors but only found t
• Unknown error
• Cannot reshape tensor with shape {tensor.shape} into {target

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/b46fdcf38d28d143. Report an issue: GitHub.