hiyouga/LlamaFactory · error · ValueError

Qwen2VL requires 3D position ids for mrope.

Error message

Qwen2VL requires 3D position ids for mrope.

What it means

LlamaFactory patches Qwen3.5 (qwen3_5 / qwen3_5_moe) GDN layers for packed-sequence (packing / neat_packing) training by delegating to flash-linear-attention kernels. _check_fla_dependencies() (src/llamafactory/model/patcher.py:64) probes for fla.modules.convolution.causal_conv1d and fla.ops.gated_delta_rule.{chunk,fused_recurrent}_gated_delta_rule and re-raises ImportError when they are missing. These symbols only exist in flash-linear-attention >= 0.4.1.

Source

Thrown at scripts/bench_qwen.py:114

        batch: dict[str, torch.Tensor] = super().__call__(features)

        batch["pixel_values"] = torch.cat(batch_pixel_values, dim=0)
        batch["pixel_values_videos"] = torch.cat(batch_pixel_values_videos, dim=0)
        batch["image_grid_thw"] = torch.cat(batch_image_grid_thw, dim=0)
        batch["video_grid_thw"] = torch.cat(batch_video_grid_thw, dim=0)

        if self.get_rope_func is not None:
            rope_index_kwargs = {
                "input_ids": batch["input_ids"],
                "image_grid_thw": batch["image_grid_thw"],
                "video_grid_thw": batch["video_grid_thw"],
                "attention_mask": (batch["attention_mask"] >= 1).float(),
            }
            batch["position_ids"], batch["rope_deltas"] = self.get_rope_func(**rope_index_kwargs)

        if "position_ids" not in batch or batch["position_ids"].dim() != 3:
            raise ValueError("Qwen2VL requires 3D position ids for mrope.")

        return batch


def bench_qwen(
    model_name_or_path: str = "Qwen/Qwen2-VL-7B-Instruct",
    batch_size: int = 1,
    seq_length: int = 2048,
    liger_kernel: bool = False,
    deepspeed_stage: int = 3,
):
    os.environ["LLAMABOARD_ENABLED"] = "true"
    os.environ["LLAMABOARD_WORKDIR"] = "output/dummy_dir"
    args = {
        "model_name_or_path": model_name_or_path,
        "enable_liger_kernel": liger_kernel,
        "stage": "sft",
        "do_train": True,

View on GitHub (pinned to f28afaf635)

Solutions

  1. pip install -U 'flash-linear-attention>=0.4.1' (builds Triton kernels; needs a matching CUDA toolchain)
  2. If you don't need packing, set `packing: false` / `neat_packing: false` in the training YAML or use a non-Qwen3.5 model so the GDN patch is never applied
  3. If you cannot build fla, switch flash_attn away from fa2 (e.g. `flash_attn: auto`) so patch_qwen3_5_forward_gpu is skipped, accepting that packing is then unavailable
  4. On NPU hardware use the triton_ascend path (patch_qwen3_5_forward_npu) instead of the CUDA fla path

Example fix

# before (YAML)
model_name_or_path: Qwen/Qwen3.5-7B
flash_attn: fa2
packing: true
# env without flash-linear-attention -> ImportError

# after
pip install -U 'flash-linear-attention>=0.4.1'
# YAML unchanged
Defensive patterns

Strategy: validation

Validate before calling

from llamafactory.extras.packages import is_flash_linear_attention_available
# or probe directly:
def can_run_qwen35_packing() -> bool:
    try:
        from fla.modules.convolution import causal_conv1d  # noqa: F401
        from fla.ops.gated_delta_rule import chunk_gated_delta_rule, fused_recurrent_gated_delta_rule  # noqa: F401
        return True
    except ImportError:
        return False
assert can_run_qwen35_packing() or not packing_enabled

Try / catch

try:
    from llamafactory.train.tuner import run_exp
    run_exp()
except ImportError as e:
    if 'flash-linear-attention' in str(e):
        raise SystemExit('Install: pip install -U \'flash-linear-attention>=0.4.1\'') from e
    raise

Prevention

When it happens

Trigger: Trainable Qwen3.5 model on CUDA with flash_attn: fa2 in the YAML config (packing/neat_packing requires fa2, see patcher.py:490-492); the environment has no flash-linear-attention or a version older than 0.4.1, so the import probe inside patch_qwen3_5_forward_gpu fails.

Common situations: Fine-tuning Qwen3.5-XXB LoRA/full with `packing: true` or `neat_packing: true` in a fresh venv or a Docker image that only installed core requirements; upgrading the repo but keeping an old pinned fla; installing fla but a CUDA/triton mismatch makes the fla package import fail entirely.

Related errors

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/c9453481c5fb7199. Report an issue: GitHub.