hiyouga/LlamaFactory · error · ValueError

Model type `{model_type}` is not supported by the Megatron B

Error message

Model type `{model_type}` is not supported by the Megatron Bridge PT/SFT path. Supported model types: {sorted(MEGATRON_BRIDGE_SUPPORTED_MODELS)}. Multimodal / audio / omni models are not enabled in v0.

What it means

Error "Model type `{model_type}` is not supported by the Megatron Bridge PT/SFT path. Supported model types: {sorted(MEGATRON_BRIDGE_SUPPORTED_MODELS)}. Multimodal / audio / omni models are not enabled in v0." thrown in hiyouga/LlamaFactory.

Source

Thrown at src/llamafactory/train/megatron_bridge/workflow.py:58

        DataArguments,
        FinetuningArguments,
        MegatronBridgeArguments,
        ModelArguments,
        TrainingArguments,
    )


logger = get_logger(__name__)


def _check_model_support(model_args: "ModelArguments") -> None:
    r"""Ensure the HF ``model_type`` is covered by the Megatron Bridge PT/SFT path."""
    config = HfAutoConfig.from_pretrained(
        model_args.model_name_or_path, trust_remote_code=model_args.trust_remote_code
    )
    model_type = getattr(config, "model_type", None)
    if model_type not in MEGATRON_BRIDGE_SUPPORTED_MODELS:
        raise ValueError(
            f"Model type `{model_type}` is not supported by the Megatron Bridge PT/SFT path. "
            f"Supported model types: {sorted(MEGATRON_BRIDGE_SUPPORTED_MODELS)}. "
            "Multimodal / audio / omni models are not enabled in v0."
        )


def _run_on_main_process(training_args: "TrainingArguments", work: Callable[[], None], sync_dir: str) -> None:
    r"""Run ``work`` only on global rank 0, then synchronize other ranks.

    Prefer ``torch.distributed.barrier`` when the process group is already initialized;
    otherwise fall back to a file flag under ``sync_dir`` so non-main ranks wait for
    shared filesystem writes (e.g. dataset export) to finish.
    """
    done_file = os.path.join(sync_dir, ".main_process_done")
    is_main = getattr(training_args, "process_index", 0) == 0
    wait_start = time.time()

    import torch.distributed as dist

View on GitHub (pinned to f28afaf635)

Solutions

1. Choose a model whose model_type is listed in MEGATRON_BRIDGE_SUPPORTED_MODELS (text-only PT/SFT models); multimodal, audio, and omni models are not supported on this path.
2. Or train without Megatron Bridge (unset USE_MEGATRON_BRIDGE).

Example fix

# use a supported text model, e.g.
model_name_or_path: meta-llama/Llama-3-8B

When it happens

Trigger: Thrown at src/llamafactory/train/megatron_bridge/workflow.py:58 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/8b4fa1a7fc037222. Report an issue: GitHub.