{"record":{"id":"c9453481c5fb7199","repo":"hiyouga/LlamaFactory","slug":"qwen2vl-requires-3d-position-ids-for-mrope","errorCode":null,"errorMessage":"Qwen2VL requires 3D position ids for mrope.","messagePattern":"Qwen2VL requires 3D position ids for mrope\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"scripts/bench_qwen.py","lineNumber":114,"sourceCode":"\n        batch: dict[str, torch.Tensor] = super().__call__(features)\n\n        batch[\"pixel_values\"] = torch.cat(batch_pixel_values, dim=0)\n        batch[\"pixel_values_videos\"] = torch.cat(batch_pixel_values_videos, dim=0)\n        batch[\"image_grid_thw\"] = torch.cat(batch_image_grid_thw, dim=0)\n        batch[\"video_grid_thw\"] = torch.cat(batch_video_grid_thw, dim=0)\n\n        if self.get_rope_func is not None:\n            rope_index_kwargs = {\n                \"input_ids\": batch[\"input_ids\"],\n                \"image_grid_thw\": batch[\"image_grid_thw\"],\n                \"video_grid_thw\": batch[\"video_grid_thw\"],\n                \"attention_mask\": (batch[\"attention_mask\"] >= 1).float(),\n            }\n            batch[\"position_ids\"], batch[\"rope_deltas\"] = self.get_rope_func(**rope_index_kwargs)\n\n        if \"position_ids\" not in batch or batch[\"position_ids\"].dim() != 3:\n            raise ValueError(\"Qwen2VL requires 3D position ids for mrope.\")\n\n        return batch\n\n\ndef bench_qwen(\n    model_name_or_path: str = \"Qwen/Qwen2-VL-7B-Instruct\",\n    batch_size: int = 1,\n    seq_length: int = 2048,\n    liger_kernel: bool = False,\n    deepspeed_stage: int = 3,\n):\n    os.environ[\"LLAMABOARD_ENABLED\"] = \"true\"\n    os.environ[\"LLAMABOARD_WORKDIR\"] = \"output/dummy_dir\"\n    args = {\n        \"model_name_or_path\": model_name_or_path,\n        \"enable_liger_kernel\": liger_kernel,\n        \"stage\": \"sft\",\n        \"do_train\": True,","sourceCodeStart":96,"sourceCodeEnd":132,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/scripts/bench_qwen.py#L96-L132","documentation":"LlamaFactory patches Qwen3.5 (qwen3_5 / qwen3_5_moe) GDN layers for packed-sequence (packing / neat_packing) training by delegating to flash-linear-attention kernels. _check_fla_dependencies() (src/llamafactory/model/patcher.py:64) probes for fla.modules.convolution.causal_conv1d and fla.ops.gated_delta_rule.{chunk,fused_recurrent}_gated_delta_rule and re-raises ImportError when they are missing. These symbols only exist in flash-linear-attention >= 0.4.1.","triggerScenarios":"Trainable Qwen3.5 model on CUDA with flash_attn: fa2 in the YAML config (packing/neat_packing requires fa2, see patcher.py:490-492); the environment has no flash-linear-attention or a version older than 0.4.1, so the import probe inside patch_qwen3_5_forward_gpu fails.","commonSituations":"Fine-tuning Qwen3.5-XXB LoRA/full with `packing: true` or `neat_packing: true` in a fresh venv or a Docker image that only installed core requirements; upgrading the repo but keeping an old pinned fla; installing fla but a CUDA/triton mismatch makes the fla package import fail entirely.","solutions":["pip install -U 'flash-linear-attention>=0.4.1' (builds Triton kernels; needs a matching CUDA toolchain)","If you don't need packing, set `packing: false` / `neat_packing: false` in the training YAML or use a non-Qwen3.5 model so the GDN patch is never applied","If you cannot build fla, switch flash_attn away from fa2 (e.g. `flash_attn: auto`) so patch_qwen3_5_forward_gpu is skipped, accepting that packing is then unavailable","On NPU hardware use the triton_ascend path (patch_qwen3_5_forward_npu) instead of the CUDA fla path"],"exampleFix":"# before (YAML)\nmodel_name_or_path: Qwen/Qwen3.5-7B\nflash_attn: fa2\npacking: true\n# env without flash-linear-attention -> ImportError\n\n# after\npip install -U 'flash-linear-attention>=0.4.1'\n# YAML unchanged","handlingStrategy":"validation","validationCode":"from llamafactory.extras.packages import is_flash_linear_attention_available\n# or probe directly:\ndef can_run_qwen35_packing() -> bool:\n    try:\n        from fla.modules.convolution import causal_conv1d  # noqa: F401\n        from fla.ops.gated_delta_rule import chunk_gated_delta_rule, fused_recurrent_gated_delta_rule  # noqa: F401\n        return True\n    except ImportError:\n        return False\nassert can_run_qwen35_packing() or not packing_enabled","typeGuard":null,"tryCatchPattern":"try:\n    from llamafactory.train.tuner import run_exp\n    run_exp()\nexcept ImportError as e:\n    if 'flash-linear-attention' in str(e):\n        raise SystemExit('Install: pip install -U \\'flash-linear-attention>=0.4.1\\'') from e\n    raise","preventionTips":["Pin flash-linear-attention>=0.4.1 in the same requirements set used for Qwen3.5 runs","Add a CI smoke test that runs `python -c \"from fla.ops.gated_delta_rule import chunk_gated_delta_rule\"` for images intended to train Qwen3.5 with packing"],"tags":["dependency","qwen3-5","packing","flash-linear-attention","installation"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}