{"record":{"id":"5cc646b9c5c8b26c","repo":"hiyouga/LlamaFactory","slug":"please-upgrade-transformers-to-4-34-0","errorCode":null,"errorMessage":"Please upgrade `transformers` to 4.34.0","messagePattern":"Please upgrade `transformers` to 4\\.34\\.0","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"scripts/convert_ckpt/llamafy_qwen.py","lineNumber":33,"sourceCode":"import json\nimport os\nfrom collections import OrderedDict\nfrom typing import Any\n\nimport fire\nimport torch\nfrom huggingface_hub import split_torch_state_dict_into_shards\nfrom safetensors import safe_open\nfrom safetensors.torch import save_file\nfrom tqdm import tqdm\nfrom transformers.modeling_utils import SAFE_WEIGHTS_INDEX_NAME, SAFE_WEIGHTS_NAME, WEIGHTS_INDEX_NAME, WEIGHTS_NAME\nfrom transformers.utils import check_min_version\n\n\ntry:\n    check_min_version(\"4.34.0\")\nexcept Exception:\n    raise ValueError(\"Please upgrade `transformers` to 4.34.0\")\n\n\nCONFIG_NAME = \"config.json\"\n\n\ndef save_weight(input_dir: str, output_dir: str, shard_size: str, save_safetensors: bool) -> str:\n    qwen_state_dict: dict[str, torch.Tensor] = OrderedDict()\n    for filepath in tqdm(os.listdir(input_dir), desc=\"Load weights\"):\n        if os.path.isfile(os.path.join(input_dir, filepath)) and filepath.endswith(\".safetensors\"):\n            with safe_open(os.path.join(input_dir, filepath), framework=\"pt\", device=\"cpu\") as f:\n                for key in f.keys():\n                    qwen_state_dict[key] = f.get_tensor(key)\n\n    llama_state_dict: dict[str, torch.Tensor] = OrderedDict()\n    torch_dtype = None\n    for key, value in tqdm(qwen_state_dict.items(), desc=\"Convert format\"):\n        if torch_dtype is None:\n            torch_dtype = value.dtype","sourceCodeStart":15,"sourceCodeEnd":51,"githubUrl":"https://github.com/hiyouga/LlamaFactory/blob/f28afaf6355af515454dfb16c97d728307c93897/scripts/convert_ckpt/llamafy_qwen.py#L15-L51","documentation":"During config patching, LlamaFactory inspects config.architectures and rejects checkpoints whose architecture list contains InternVLChatModel (patcher.py:404). The original OpenGVLab InternVL releases ship a custom InternVLChatModel implementation that is incompatible with LlamaFactory's transformers-based training/inference path, so it demands a checkpoint converted to the HF-native InternVLForCausalLM format.","triggerScenarios":"model_name_or_path points at a raw OpenGVLab release such as OpenGVLab/InternVL3-8B or a local copy of it; AutoConfig reports architectures containing 'InternVLChatModel' and _check_audio_module / config patching raises ValueError before any weights load.","commonSituations":"Downloading the original model card checkpoint instead of the '-hf' variant; fine-tuning a third-party InternVL finetune that was published in the legacy format; pointing at an old local InternVL2/InternVL checkpoint directory.","solutions":["Use an HF-converted checkpoint, e.g. model_name_or_path: OpenGVLab/InternVL3-8B-hf (any *-hf InternVL repo)","If you must keep the original weights, convert the checkpoint to HF format (transformers conversion scripts) before training"],"exampleFix":"# before\nmodel_name_or_path: OpenGVLab/InternVL3-8B  # legacy InternVLChatModel format\n\n# after\nmodel_name_or_path: OpenGVLab/InternVL3-8B-hf","handlingStrategy":"validation","validationCode":"from transformers import AutoConfig\ncfg = AutoConfig.from_pretrained(model_path, trust_remote_code=False)\narch = getattr(cfg, 'architectures', []) or []\nassert 'InternVLChatModel' not in arch, 'Use an HF-format InternVL checkpoint, e.g. OpenGVLab/InternVL3-8B-hf'","typeGuard":"def is_hf_internvl(model_name_or_path: str) -> bool:\n    cfg = AutoConfig.from_pretrained(model_name_or_path, trust_remote_code=False)\n    archs = getattr(cfg, 'architectures', None) or []\n    return 'InternVLChatModel' not in archs and any('InternVL' in a for a in archs)","tryCatchPattern":"try:\n    run_sft(train_args)\nexcept ValueError as e:\n    if 'InternVL' in str(e):\n        raise SystemExit('Switch to an -hf InternVL checkpoint (OpenGVLab/InternVL3-8B-hf)') from e\n    raise","preventionTips":["Standardize on *-hf converted multimodal checkpoints in dataset/model configs","Script a pre-flight check over config.architectures for InternVLChatModel/LlavaLlamaForCausalLM before launching long jobs"],"tags":["model","internvl","multimodal","checkpoint-format","config"],"backgroundTag":null,"analyzedSha":"f28afaf6355af515454dfb16c97d728307c93897","analyzedAt":"2026-08-14T21:57:28.298Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}