{"record":{"id":"b46fdcf38d28d143","repo":"huggingface/transformers","slug":"permuteforrope-expects-a-single-tensor-per-key","errorCode":null,"errorMessage":"PermuteForRope expects a single tensor per key.","messagePattern":"PermuteForRope expects a single tensor per key\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/transformers/core_model_loading.py","lineNumber":462,"sourceCode":"    def convert(\n        self,\n        input_dict: dict[str, list[torch.Tensor] | torch.Tensor],\n        source_patterns: list[str],\n        target_patterns: list[str],\n        config,\n        **kwargs,\n    ) -> dict[str, list[torch.Tensor]]:\n        self.config = config\n        output: dict[str, list[torch.Tensor]] = {}\n        for key, tensors in input_dict.items():\n            # Permute q and key weights back (skip biases) to match original RoPE implementation\n            if not any(name in key for name in self.permute_layer_names):\n                output[key] = tensors\n                continue\n\n            if isinstance(tensors, list):\n                if len(tensors) != 1:\n                    raise ValueError(\"PermuteForRope expects a single tensor per key.\")\n                tensors = tensors[0]\n            output[key] = self._apply(tensors)\n        return output\n\n    @property\n    def reverse_op(self) -> ConversionOps:\n        return PermuteForRope(\n            subconfig_key=self.subconfig_key, permute_layer_names=self.permute_layer_names, inverse=not self.inverse\n        )\n\n\nclass VisionFuseAndPermuteForRope(ConversionOps):\n    \"\"\"\n    Applies the permutation required to convert complex RoPE weights to the split sin/cos format on fused QKV.\n    Same as calling `PermuteForRope() + Concatenate()` but lets us call `Permute` only on a subset of chunked tensors.\n\n    NOTE: this conversion applies only to a vision backbone in multimodal models, because it checks `config.vision_config`\n    \"\"\"","sourceCodeStart":444,"sourceCodeEnd":480,"githubUrl":"https://github.com/huggingface/transformers/blob/a597f974857b3d92939971296bc0deb93d33d780/src/transformers/core_model_loading.py#L444-L480","documentation":"Raised by PermuteForRope.convert (core_model_loading.py:462). This op permutes q/k weight matrices to move between complex-RoPE layout and split sin/cos layout; the permutation is only defined for a single 2D weight per key. If a key matched by permute_layer_names arrives as a list holding zero or more than one tensor (e.g. because several source checkpoint keys were fused/stacked into that target key), the op cannot know which tensor to permute and aborts.","triggerScenarios":"A PermuteForRope(subconfig_key=..., permute_layer_names=[...]) op where one of the matched keys maps to a list of tensors with len != 1 — typically when an earlier op in the chain (e.g. Fuse) produced stacked tensors under that key, or when the rename layer targets both a weight and something else under one key.","commonSituations":"Conversion recipes for Llama-family / RoPE-based models (or vision towers using RoPE, e.g. Qwen-VL style) where q_proj/k_proj keys are expected single tensors, but the checkpoint or a preceding conversion step delivered fused or multiple shards (e.g. multi-GPU sharded checkpoints collected under one key).","solutions":["Debug what input_dict contains for the failing key: if a list, find out which prior op fused/sharded it and adjust ordering or patterns so PermuteForRope sees one plain tensor per key.","Restrict permute_layer_names so it only matches actual q/k weight names (e.g. 'q_proj' and 'k_proj'), not keys that carry tensor lists.","If shards are involved, merge/concatenate them before the permute step in the operations chain."],"exampleFix":"# before: pattern also matches fused list tensors\nPermuteForRope(subconfig_key=\"text_config\", permute_layer_names=[\"q\", \"k\", \"gate\"])\n\n# after: only single-tensor q/k weights\nPermuteForRope(subconfig_key=\"text_config\", permute_layer_names=[\"q_proj\", \"k_proj\"])","handlingStrategy":"validation","validationCode":"for key, tensors in collected.items():\n    if any(name in key for name in permute_layer_names):\n        assert not isinstance(tensors, list) or len(tensors) == 1, (\n            f'PermuteForRope key {key} carries {len(tensors)} tensors; expected 1'\n        )","typeGuard":"def is_single_tensor_entry(entry) -> bool:\n    return (not isinstance(entry, list)) or len(entry) == 1","tryCatchPattern":null,"preventionTips":["Restrict permute_layer_names to exact q/k weight identifiers.","Order operations so fuse/stack steps run after the permute, not before.","When converting sharded checkpoints, merge shards before applying PermuteForRope."],"tags":["weight-conversion","rope","permutation","model-loading"],"backgroundTag":null,"analyzedSha":"a597f974857b3d92939971296bc0deb93d33d780","analyzedAt":"2026-08-14T18:24:08.354Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}