hiyouga/LlamaFactory · error · ValueError

Merged position_ids shape mismatch: got {features['position_

Error message

Merged position_ids shape mismatch: got {features['position_ids'].shape}, expected {expected_position_ids_shape}.

What it means

During multimodal feature merging, after concatenating the dummy-image right-padding onto position_ids, the code compares features['position_ids'].shape against a precomputed expected_position_ids_shape and raises ValueError on mismatch. A mismatch means the mrope position computation produced a sequence length inconsistent with the (possibly padded) input_ids/attention_mask — almost always an image-token-count disagreement between the template's placeholder tokens and what the mm_plugin actually expanded.

Source

Thrown at src/llamafactory/data/collator.py:321

        expected_position_ids_shape = (
            (bsz, seq_len)
            if all_position_ids[0].dim() == 2
            else (
                all_position_ids[0].size(0),
                bsz,
                seq_len,
            )
        )
        # Check if position_ids shape matches expected shape.
        # for further usage, we should padding to the right when some padding token on the right.
        if has_dummy_image:
            features["position_ids"] = torch.cat([features["position_ids"], dummy_image_right_padding_mrope], dim=-1)
            features["attention_mask"] = torch.cat(
                [features["attention_mask"], dummy_image_right_padding_attention_mask], dim=-1
            )

        if features["position_ids"].shape != expected_position_ids_shape:
            raise ValueError(
                "Merged position_ids shape mismatch: "
                f"got {features['position_ids'].shape}, expected {expected_position_ids_shape}."
            )

    def __call__(self, features: list[dict[str, Any]]) -> dict[str, "torch.Tensor"]:
        model_type = getattr(getattr(self.model, "config", None), "model_type", None)
        is_moss_vl = model_type == "moss_vl"
        batch_images, batch_videos, batch_audios = [], [], []
        batch_imglens, batch_vidlens, batch_audlens, batch_input_ids = [], [], [], []
        packing_params_list: list[dict[str, Any] | None] = []
        for feature in features:
            images = feature.pop("images", None) or []
            videos = feature.pop("videos", None) or []
            audios = feature.pop("audios", None) or []
            batch_images.extend(images)
            batch_videos.extend(videos)
            batch_audios.extend(audios)
            batch_imglens.append(len(images))

View on GitHub (pinned to f28afaf635)

Solutions

Verify you are using the template registered for your model family (e.g. qwen2_vl / qwen2_5_vl), not a generic one.
Pin/downgrade transformers to a version known compatible with your LlamaFactory release — image token counting changed across 4.49-4.52.
Re-run data preprocessing (clear the cached tokenized dataset) after changing template or processor settings.
If it happens only on specific samples, inspect those images (corrupt, extreme aspect ratio) and drop or fix them.
Report with full repro if it persists on stock configs — the padding bookkeeping in the dummy-image path may need a fix.

Defensive patterns

Strategy: validation

Validate before calling

# preflight one batch before training
collator = MultiModalDataCollatorForSeq2Seq(tokenizer=tokenizer, model=model, template=template)
sample = collator([next(iter(train_dataset))])
assert "position_ids" not in sample or sample["position_ids"].shape[-1] == sample["input_ids"].shape[-1]

Try / catch

try:
    batch = collator(features)
except ValueError as e:
    if "position_ids shape mismatch" in str(e):
        # dump the offending sample ids for triage, then abort run
        logger.error("mrope mismatch on batch; check template/transformers pairing")
        raise

Prevention

Lock the (LlamaFactory, transformers) version pair that worked for your VLM.
Clear tokenization caches after any template/processor change.
Run a 1-batch dry run through the collator before long jobs.

When it happens

Trigger: Training a VLM (qwen2-vl/qwen2.5 style mrope models) where the number of <image> expansion tokens in the encoded example differs from what the processor returns (e.g. image processed at a different resolution/grid, dummy image injection with has_dummy_image, or a template not designed for the model's mm token counting).

Common situations: Custom or mismatched template for qwen2-vl family; transformers version change altering image token expansion counts; enabling has_dummy_image paths on batches whose attention_mask was padded differently; corrupt image files producing zero-size patches.

Related errors

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/39b79f279b2410ec. Report an issue: GitHub.